
GRAPHICAL- INTERACTIVE-SCREEN DISPLAY APPARATUS 
AND PERIPHERAL UNITS 

BACKGROUND OF THE INVENTION 

(1) Field of the Invention 

The present invention relates to a display apparatus 
showing graphical interactive screens for remote control, and 
to the peripheral units. 

(2) Description of the Prior Art 

The use of remote controllers has spread to facilitate 
the manipulation of today's household appliances such as audio- 
visual apparatuses, and one remote controller is furnished for 
one appliance in general. However, it is quite cumbersome to 
manage a plurality of remote controllers. 

In addition, a remote controller in a broad sense, or 
a display on the screen that enables the users to make a 
variety of selections, was developed and has been in practical 
use for the interactive TV (television) system and TV game 
players. However, this type of remote controller also faces 
problems such as facilitating manipulation and saving 
manufacturing cost. 

To eliminate the above problems, a multi-remote 
controller, or otherwise known as a system remote controller, 
was developed and has been commercially available. The multi- 
remote controller includes a transmitter that controls a 



plurality of appliances of various types in one kind (for 
example, TVs made by a plurality of makers), or various kinds 
of appliances (for example, TV and VTR (video tape recorder)), 
so that the user can control a plurality of appliances with one 
5 multi-remote controller by selecting the appliance he wishes 
to control. Alternately, the multi-remote controller may 
include the buttons necessary for the manipulation of all the 
appliances. However, in either case, the user must select a 
desired appliance first, meaning that the multi -remote 

10 controller must include the selection button. 

To compensate the drawback of the multi-remote 
controller, Japanese Laid-Open Patent Application No. 2-180497 
discloses a remote controller comprising a remote control unit 
for transmitting remote control signals for a plurality of 

15 appliances to be controlled, and receiver units furnished with 
each appliance for receiving the remote controller signals for 
their respective appliances. Note that each receiver" unit 
transmits a unique signal to the remote controller unit, so 
that the receiver unit stores data as to which control signal 

20 should be transmitted to which appliance, eliminating the 
appliance selection process and hence the selection button. 

However, this remote controller is available only to the 
predetermined appliances and functions, and it must include a 
ultra red ray signal transmitter and buttons for each 

25 predetermined function. Thus, although the selection button 
can be omitted, the remote controller still includes a large 



number of buttons if it controls more than one kind of 
appliance (for example, TV and VTR), because the buttons for 
channel selection and volume adjustment for the TV receiver, 
and those for playback and record functions for the VTR must 
5 be furnished separately. Further, the design of the remote 
controller can not be changed flexibly, and unnecessary buttons 
under a particular situation are still furnished and the 
control signals for such buttons are stored in the memory as 
well. Thus, neither the design is adequate nor the memory is 

10 utilized efficiently, complicating the manipulation of the 
remote controller more than necessary. 

Also, a TV- VTR compound apparatus, which is controlled 
by menu information defined in .advance, has been developed 
However, it is impossible to control a plurality of appliances 

15 as an integral system; for the menu information varies 
depending on the state of each appliance, such as 
interconnection and power ON/OFF. 

Further, a menu processing method, disclosed in Japanese 
Laid-Open Patent Application No. 3-233660, was developed. This 

20 method enables the retrieval of a desired application program 
by switching the menu displays regardless of which terminal or 
host computer stores the application program in a computer 
environment where a plurality of terminals are interconnected 
via a network. 

25 However, the information as to which terminal or host 

computer stores the secondary and subsequent order menus must 



be written into each item in "the menu in advance. Thus, it 
is impossible to control a plurality of appliances as an 
integral, system; for the menu information varies depending on 
the state of each appliance, such as interconnection and power 
5 ON/OFF. 

In addition, Japanese Laid-Open Patent Application No, 
3-172086 discloses a program receiver which can display texts 
and graphics for the operation manual or guidance of the CATV 
(Cable TV) system, either related or unrelated to the program 

10 being broadcasted, on the screen. This will be explained more 
in detail while referring to FIG. 1 depicting the structure of 
the program receiver. A terminal 101 includes a character 
generator 103 for generating texts and graphics displayed on 
a screen 102, a memory 104 storing all the patterns for the 

15 texts and graphics transmitted from a head-end of the CATV 
station in advance, a switcher 107 for inputting signals from 
the character generator 103 or memory 104 into a TV receiver 
106 instead of or together with the outputs from a tuner 105. 
The text and graphical information, or namely the operation 

20 manual or guidance and the information either related or 
unrelated to the program being broadcasted, is transmitted 
from the head end in advance to upgrade the utility of the CATV 
system. 

In this system, program distributors supply the 
25 interactive screen generation information for each program to 
the CATV station in advance, so that the information is 



-transmitted from the head-end to the memory 104 each time a 
program is switched, supplying an adequate interactive screen 
for each program. However, this complicates the re- 
transmission of the program. As well, each time the user 
switches a channel, the head-end detects the switching and 
transmits the interactive screen generation information for the 
switched program to the memory 104, making the process at the 
head-end cumbersome. 

In addition, a technique such that enables the user to 
set the functions of the VTR and TV game players by using the 
display on the TV receiver was developed. The household 
appliances of this type include a circuit for generating a user 
interface screen, and a terminal for outputting image signals 
generated by the circuit to an external appliances. The 
external output terminal is connected to an external image 
signal input terminal of the TV receiver via an image signal 
cable to switch TV receiver f s signal input to the external 
image signal input terminal. Thus, the TV receiver displays 
the input image signal from the external image signal input 
terminal, supplying the user interface screen to the user. 

The graphical display apparatus used for CAD (Computer 
Aided Design) also employs the user interface screen. Here, 
the host computer transmits figure definition instructions: 
combination and display position of basic elements (segments, 
arcs, characters, etc.), and attribution specification such as 
display allowance or disallowance. The graphical display 



apparatus interprets the figure definition instructions and 

i 

then displays figures as per instructions oh the screen. The 
definition of a figure composed of the same jbasic elements can 
be* given only by copying the definition of the element and 
changing the display position attribute. , Thus, the figure 
definition instructions for one element is transmitted from the 
host computer only once, saving the information transmission 
time for the host computer. However, in case of a multi- layer 
structure such as a menu manipulation screen to switch a layer 
as per user's instruction, the screen is cleared (the defined 
figure data are deleted) each time a layer is switched, and all 
the figure definition instructions are transmitted again for 
the switched layer. Alternately,, the figure data for all the 
layers may be defined in advance, and an instruction is sent 
each time a layer is switched to disallow the display of the 
attribute of the figure data in the currently displayed layer, 
and another instruction is sent to allow the display of the 
figure data for the switched layer. : 

However, using the user interface screen with the 
household appliances such as the VTR causes the following 
problems : 

(1) the user-interface-screen generation circuit must 
be furnished for each appliance, causing the manufacturing 
cost to increase. Thus, this technique is applied only to 
limited appliances. 

i 

f (2) the image input for the TV receiver must be switched 



\ i 



to; the external image signal input. This is; not a drawback for 
the VTR or TV game players activated by switching the image 
input to the external image signal input. However, it is quite 
[inconvenient if functions for an air conditioner or the like 

arfe set in this manner. Although such inconvenience can be 

j 

eliminated by furnishing the user interface screen to the air 

i 

conditioner, it costs more than furnishing the user-interface- 
screen generation circuit. * 

(3) the appliances and TV receiver must be connected via 

i 

the image signal cable. Since the image signal cable is an 
essential component for the VTR and TV game player, this is not 
drucial for such appliances. However, this is quite 
inconvenient if the appliances, which can be: used independently 
of the image signal cable, are controlled in this manner. 
Although such inconvenience can be eliminated by furnishing the 
user interface screen, it costs more than furnishing the user- 
interface-screen generation circuit as was with the secondly 
mentioned problem. ! 

(4) the graphical display apparatus is connected to the 
host computer in the level of the basic elements to realize a 
variety of displays, thus many a figure definition instruction 
is received from the host computer when switching the display. 
This causes to increase the load in the host computer, and 

makes the real time transmission impossible if a high- 

j 

transmission rate communication route is employed. More 

11 

precisely, micro computers for the household appliances are 



less efficient compared with the one for host computer, and the 
transmission rate is set relatively low to save the 

transmission cost. Thus, it is not easy to display the 

\ 

graphical interactive screens on the graphical display 

j 

apparatus using the display information. • 

I 

SUMMARY OF THE INVENTION 

Accordingly, the present invention has a first object 
to provide a remote controller which is easy to manipulate and 
does not hold the information of the appliances under the 
control thereof in advance, and thus uses resources such as a 
memory efficiently* 

The present invention has .a second object to provide a 
remote controller whose design can be ( changed flexibly 
depending on the state of the appliances \ under the control 
thereof. 

V. The present invention has a third object to provide a 

remote controller which can synthesize the control menus for 
a plurality of appliances under the control thereof. 

The present invention has a fourth object to provide a 
program receiver which can supply an adequate interactive 
screen for each program to the user. 

The present invention has a fifth object to provide a 
graphical -interactive-screen display apparatus and the 
peripheral units which can display the user interface screens 
without using expensive user-interface-screen generation 

8 



circuit:, so that the user sets the functions for a desired 
appliance using such inexpensive user interface screens. 

The first object can be fulfilled by a remote controller 
comprising a structure information supply^ unit for holding 
structure information for various types j of appliances in 
advance, and a portable remote control unit for controlling 

1 

the appliances, (1) the structure information supply unit 
including: a transmitter's structure information storage unit 
for storing the structure information, the structure 
information including control information for each appliance 
and object information, the control information being used by 
the remote control unit in controlling the appliances, the 
object information representing manipulation objects forming 
a manipulation unit of the remote control unit; and a structure 
information transmission unit for retrieving the control 
information and the object information from the transmitter's 
structure information storage unit to send the retrieved 
information to the remote control unit, ( 2 ) the remote control 
unit including: a structure information, receipt unit for 
receiving the control information and the object information 
from the structure information transmission unit; a receiver's 
structure information storage unit for storing the control 
information and the object information received by the 
structure information receipt unit, a display unit for 
displaying a picture having a manipulation area used for 
manipulating the appliances; and a first display control unit 



for controlling the display unit to plac^ the manipulation 
objects on the manipulation area based on the object 
information stored in the receiver's structure information 
storage unit, < 

The remote control unit may further include: a 
manipulation detection unit for detecting which manipulation 
object is manipulated on the manipulation area; a first control 
signal transmission unit for transmitting a control signal 
assigned for the manipulation object ^detected by the 
manipulation detection unit to the appliances based on the 
control information stored in the receiver's structure 
information storage unit, whereby the appliances operate in 
accordance with the control signal from the remote control 
unit . 

The structure information supply unit may be installed 
in each appliance controlled by the control signal from the 
remote control unit. 

To fulfill the second object, in the. remote controller, 
the structure information supply unit may further include: a 
structure information generation unit for' generating state- 
based control information depending on an action state of each 
appliance, the state-based control information being 
transmitted to the remote control unit < by the structure 
information transmission unit; an object information generation 
unit for generating the object information depending on the 
action state of each appliance, the object information being 

10 ! 



transmitted to the remote control unit \ by the structure 
information transmission unit, and the remote control unit may 
further include: a generated information, receipt unit for 
receiving the state-based control information from the 
structure information transmission unit; a generated 
information update unit for updating the control information 
and the object information in the receiver 1 s structure 
information storage unit in accordance with the state-based 
control information received by the generated information 
receipt unit; a second control signal transmission unit for 
transmitting a control signal assigned 1po the manipulated 
object detected by the manipulation detection unit to the 
appliances based on updated control information stored in the 
receiver's structure information storage unit; a generated 
object information receipt unit for receiving updated object 
information from the structure information transmission unit; 
and a second display control unit for controlling the display 
unit to place updated manipulation objects on the manipulation 
area based on the updated object information received by the 
/generated object information receipt unit., 

The third object can be fulfilled by a synthesis 
apparatus including a plurality of appliances for synthesizing 
menu information withheld by each appliance, the menu 
information being hierarchical information enabling a user to 
control the appliances interactively, (1) one of the plurality 
of appliances including: a first menu storage unit for storing 



first menu information, the first menu ^information being 
hierarchical information to be used to J control the one 
.appliance; a menu information receipt unit* for receiving the 
menu information from the other appliances; ;a menu information 
'synthesis unit for synthesizing the menu information received 
% by the menu information receipt unit and first menu information 
stored in the first menu information storage unit, resulting 
synthesized menu information being used as a menu-information 
selection menu at a highest position in a hierarchy; and a menu 
display unit for displaying the menu-information selection menu 
generated by the menu information synthesis unit, (2) each of 
the other appliances including: a menu storage unit for storing 
the menu information unique to each appliance, the menu 
information being hierarchical information to be used to 
control each appliance; and a menu transmission unit for 
transmitting the unique menu information in the menu storage 
unit to the one appliance. 

The one appliance may further include a synthesis rule 
storage unit for storing rules regulating menu-information 
synthesis by the menu information synthesis unit, whereby the 
menu information synthesis unit synthesizes the menu 
information from the other appliances and the first menu 
information in accordance with the synthesis rules. 

The menu transmission unit may include: a menu 
information retrieval direction unit for detecting a connection 
with the one appliance and for giving a direction to retrieve 

12 



the menu information; a menu information retrieval unit for 

•i 

retrieving the menu information from the menu storage unit upon 

j 

receipt of the direction from the menu information retrieval 
direction unit; and a menu transmission unit for transmitting 
the menu information retrieved by the ) menu information 
retrieval unit to -the menu information receipt unit:. 

The menu information synthesis unit may include: a menu 
selection acceptance unit for accepting a selection of an 
appliance subject to control manipulation using the menu- 
information selection menu displayed by the menu display unit; 
a menu information retrieval unit for retrieving the menu 
information of the appliance accepted by 'the menu selection 
acceptance unit from the menu information synthesis unit; and 
a menu switch control unit for changing a display on the menu 
display unit in accordance with the menu information retrieved 
by the menu information retrieval unit. 

The menu information synthesis unit: may further include: 
a menu information temporarily storage unit for temporarily 
storing the menu information received by the menu information 
receipt unit; a first menu information retrieval unit for 
retrieving the first menu information from the first menu 
storage unit; a synthesis rule retrieval unit for retrieving 
the synthesis rules from the synthesis rule storage unit, the 
synthesis rules including a synthesis rule for a hierarchial 
' structure, a selection rule for a displayed shape, a synthesis 
rule for a panel object, and a synthesis rule for control of 

13 



the appliances; and a hierarchial structure 4 synthesis unit for 

j 

synthesizing the menu information stored in the menu 
information temporarily storage unit an<3 the first menu 
information retrieved by the first menu information retrieval 
unit in accordance with the hierarchical synthesis rule 

Retrieved by the synthesis rule retrieval Unit. 

i 

The menu information synthesis unit may further include: 
a menu information temporarily storage unit for temporarily 
storing the menu- information received by the menu information 
receipt unit; a first menu information retrieval unit for 
retrieving the first menu information from the first menu 
storage unit; a synthesis rule retrieval unit for retrieving 
the synthesis rules from the synthesis rule storage unit, the 
synthesis rules including a synthesis rule for a hierarchial 
structure, a selection rule for a displayed 1 shape, a synthesis 
rule for a panel object, and a synthesis rule for control of 
the appliances; and a hierarchial structure synthesis unit for 
syhthesizing the menu information stored in the menu 
information temporarily storage unit and the first menu 
information retrieved by the first menu information retrieval 
unit in accordance with the hierarchical synthesis rule 
retrieved by the synthesis rule retrieval unit. 

The third object also can be fulfilled by in a remote 
control system comprising a plurality of appliances and a 
remote controller for transmitting control signals to the 
plurality of appliances, (1) the remote controller including: 

14 



I 

f 

a menu information request transmission unijt for transmitting 

i 

a menu information request to each appliance, the menu 

i 

information being hierarchial information unique to each 

{ * 

appliance for enabling a user to control each appliance 
interactively; a menu receipt unit for receiving the menu 
information transmitted from each appliance;, a menu information 
synthesis unit for synthesizing the menu information received 
by the menu receipt unit; a menu display unit for visually 
displaying synthesized menu information generated by the menu 
information synthesis unit; an input manipulation unit for 
accepting a user 1 s input manipulation : to control the 
appliances; and a control signal transmission unit for 
transmitting the control signals to the appliances based on the 
input manipulation accepted by the input ;manipulation unit, 
(2) each appliance including: a menu information request 
receipt unit for receiving the menu information request from 
the menu information request transmission unit; a menu 
information transmission unit for transmitting the menu 
information to the menu receipt unit when the menu information 
request receipt unit receives the menu information request; a 
control signal receipt unit for receiving the control signals 
from the control signal transmission unit; and a control unit 
for controlling the appliances as per control signals received 
by the control signal receipt unit. , 

The menu information synthesis unit may include: a menu 
information storage unit for storing the menu information of 

15 



each appliance received by the menu receipt unit, the menu 

information being hierarchical information;' including a panel 

« j 
object name and shape information; a synthesis rule storage 

unit for storing synthesis rules regulating menu-information 

synthesis, the synthesis rules including a jsynthesis rule for 

a hierarchial structure, a synthesis rule for a panel object, 

a selection rule for a displayed shape, and a synthesis rule 

for control of the appliances; a menu synthesis unit for 

synthesizing the- menu information of the appliances stored in 

the menu information storage unit into; one hierarchical 

structure in accordance with the hierarchical synthesis rules 

in the synthesis rule storage unit; a panel object synthesis 

unit for synthesizing the menu information of the appliances 

in the menu information storage unit into One panel object in 

accordance with the synthesis rule for the panel object stored 

in the synthesis rule storage unit, the synthesis rule for the 

panel object being to synthesize the menu information 

identified by an identical panel object name; a shape synthesis 

unit for making the shape information uniform for . each 

appliance for the menu information in the menu information 

storage unit in accordance with the selection rule for the 

displayed shape in the synthesis rule storage unit, the 

selection rule being to select the shape information of a 

specified appliance and apply the same to the shape information 

for each appliance for the menu information in the menu 

information storage unit; and an action append unit for 

16 



j 

i 

inserting data as to an control action in accordance with the 
synthesis rule for the control of the appliances, the synthesis 

rule for the control being to insert the data as to the control 

i 

action described by the synthesis rule fot the control at a 
position specified by the synthesis rule for the control. 

The menu display unit may include: a bit map data 
conversion unit for converting the menu information synthesized 
by the menu synthesis unit, panel synthesis unit, shape 
synthesis unit, -and action append unit into bit map data for 
a visual display; a display unit for displaying the menu 
information converted into the bit map data by the bit map data 
conversion unit; a first manipulation acceptance unit for 
accepting a user's selection of menu displayed by the display 
unit; and a selected menu notice unit for notifying a menu 
selected and accepted by the first manipulation acceptance 
unit to the menu information synthesis unit. 

The menu information synthesis unit may further include 
a menu information retrieval unit for retrieving the menu 
information from the menu storage unit for the appliance 
corresponding to the menu notified by the selected menu notice 
unit, retrieved menu information including information as to 
manipulation objects displayed by the display unit, and the 
menu display unit may further include: a menu information 
conversion unit for converting the menu information retrieved 
by the menu information retrieval unit int6 the bit map data, 
the menu information retrieved by the; menu information 

17 1 



retrieval unit including information as I to a display of 

l 

manipulation objects displayed by the displjay unit; a change- 

menu display control unit for controlling t,he display unit to 

i 

i display the menu inf ormatioif converted int<p the bit map data 



by the menu information conversion unit; a second manipulation 
acceptance unit for accepting a user's control manipulation 
related to a control over the appliance using the manipulation 
objects displayed by the display unit; and a manipulation 
notice unit for notifying the control manipulation accepted by 



-the second manipulation acceptance unit to 



the control sinal 



transmission unit. 

f 

' The fourth object can be fulfilled by a program receiver 
for displaying a graphical interactive picture by receiving a 
program transmitted from a program transmitter, the program 
receiver comprising: a storage unit for storing a plurality of 
basic picture elements in advance, the plurality of basic 
picture elements being figures composing the graphical 
interactive picture manipulated by a user; a signal receipt 
unit for receiving a signal transmitted ' from the program 
transmitter, the signal being a multiplex Isignal including a 
program and data specifying a structure; of the graphical 
interactive picture; a signal separation unit for separating 
the signal received by the signal receipt unit into a program 
signal and a graphical-interactive-picture-structure 

specification data signal; a first graphical interactive 

j 

picture generation unit for generating the graphical 

is i 



interactive picture by combining the basic picture elements 
stored in the storage unit based on the graphical-interactive- 
picture- structure specification data signal from the signal 

) 

separation unit; and a display unit for displaying the 

•' s 
graphical interactive picture generated by the first graphical 

interactive picture generation unit. « 

The program receiver may further comprise: an 

interactive manipulation unit for inputting manipulation to the 

graphical interactive picture displayed by. the display unit; 

a basic action storage unit for storing; a content of an 

action for updating the graphical-interactive-picture- 

i 

{Structure specification data; and a : second graphical 
interactive picture generation unit for retrieving the content 

of the action from the basic action storage unit based on 

i 

action information directing an update j of the graphical 
interactive picture upon receipt of the input manipulation from 
the interactive manipulation unit to update the graphical- 
interactive-picture-structure specification data to generate 

i 
i 

an updated graphical interactive picture. ! 

i 

The basic picture elements in the storage unit may be 

\ 

composed of file names identifying each ba^ic picture element 

i 

and the bit map data for each basic picture element, and the 

first graphical interactive picture generation unit may 

\ 
\ 

include: a receipt • decode unit for receiving the graphical- 
interactive-picture-structure specification data signal from 
the signal separation unit to decode the same; a storage unit 

19 



for storing decoded graphical-interactive-picture-structure 
specification data from the receipt • decode unit, the graphical- 

interactive-picture-structure specif ication.data being composed 

j 

of/ a class definition including a plurality pieces of class 

' 5 attribute information, a panel object definition including a 

! 

/ plurality pieces of panel object information, an action 

> definition including a plurality pieces of action information, 

* ' i 

and a shape definition including a plurality pieces of shape 

i 

information; a first process unit for retrieving the graphical- 
10 interactive-picture-structure specification data from the 

storage unit, extracting the basic j picture elements 
corresponding to the file names by referring to the shape 
information from the storage unit in accordance with the 
retrieved graphical-interactive-picture-structure specification 
15 data, and for placing the extracted basic picture elements by 
referring to the panel object information; and a first display 

i 

control unit for controlling the display unit to display the 

\ 

basic picture elements placed by the first process unit as the 
graphical interactive picture. ! 

20 The interactive manipulation unit may include: an input 

I 

manipulation acceptance unit for accepting a user's input 

i 

manipulation to the graphical interactivb picture; and an 

! 

interactive signal transmission unit for transmitting the input 

i 

manipulation accepted by the manipulation j acceptance unit to 

l 

25 the second graphical interactive picture generation unit as an 
interactive signal, and the second graphical interactive 

20 



picture generation unit may include: an interactive signal 
receipt unit for receiving the interactive signal from the 
interactive signal transmission unit; an interactive signal 
interpretation unit for interpreting the interactive signal 
received by the interactive signal receipt ; unit; a graphical- 
interactive-picture-structure specification data update unit 
for retrieving a content of an action from the basic action 
storage unit in accordance with the interactive signal 
interpreted by the interactive signal interpretation unit to 
update the graphical - interact ive- picture- structure 
specification data in the storage unit; a second process unit 
for retrieving updated graphical-interactive-picture-structure 
specification data from the storage unit, f and for extracting 

the basic picture elements corresponding to the file names from 

i 

the storage unit to place the extracted display elements; and 
fa second display control unit for controlling the display unit 
* to display the basic picture elements placed by the second 

process unit as an updated graphical interactive picture. 

The program receiver may further compirise an information 

transmission unit for transmitting the data of the graphical 

interactive picture updated by the interaptive manipulation 

unit to the program transmitter. j 

The basic picture elements in the storage unit may be 

i 

composed of file names identifying each ba^ic picture element 

j 

and the bit map data for each basic picturje element, and the 
first graphical interactive picture generation unit may 

I 



include: a receipt • decode unit for receiving the graphical- 
interactive-picture-structure specification; data signal from 

the signal separation unit to decode the same; a data division 

i 

unit for dividing the decoded graphical-interactive-picture- 
Structure specification data from the receipt • decode unit into 
a first graphical-interactive-picture-structure specification 
data and a second graphical-interactive-picture-structure 
specification data, the first graphical-interactive-picture- 
structure specif ication data excluding a synthesis instruction, 
the second graphical-interactive-picture- structure 
specification data including the synthesis instruction, the 
graphical-interactive-picture-structure specification data 
being composed of a panel object definition including a 
plurality pieces of panel object information, an action 
definition including a plurality pieces of action information, 
a shape definition including a plurality pieces of shape 
information, and synthesis direction data including a synthesis 
command in case of the second graphical-interactive-picture- 
structure specification data; a first graphical-interactive- 
picture- structure specification data storage unit for storing 

the first graphical-interactive-picture-structure specification 

\ 

/data; a second graphical-interactive-picture-structure 

l 

specification data storage unit for storing the second 

I 

graphical-interactive-picture-structure specification data; a 

\ 

graphical interactive picture synthesis unit for synthesizing 
the first graphical-interactive-picture-structure specification 

22 



data in the first graphical-interactive-picture-structure 
specification data storage unit and the second graphical- 
interactive-picture-structure specification: data in the second 

i 

graphical- interactive-picture- structure specification data 
storage unit in accordance with the synthesis command of the 
synthesis direction data stored in the second graphical - 
interactive-picture-structure specification data storage unit; 
a graphical-interactive-picture-structure specification data 
update unit for, updating the first graphical-interactive- 
picture-structure specification data in the first graphical- 
interactive-picture-structure specification data storage unit 
with the synthesized graphical-interactive-picture-structure 
specification data from the graphical interactive picture 
synthesis unit; a first process unit for retrieving updated 
graphical-interactive-picture-structure specification data from 
the first graphical-interactive-picture-structure specification 
data storage unit, and for extracting the basic picture 
elements corresponding to the file names fry referring to the 
shape information in the storage unit to palace the extracted 
basic picture elements by referring to ! the panel object 
information; and a first display control uiiit for controlling 

the display unit to display the basic picture elements placed 

* i 

l 

by the first process unit as an updated graphical interactive 
picture. j 

The program receiver may further comprise an information 

record unit for outputting data related! to the graphical 

j 

23 j 
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interactive picture as per manipulation form the interactive 
manipulation unit to make a record thereof. 

The fifth object can be fulfilled by a system including 
peripheral appliances holding graphical interactive picture 
structure data and a display unit for displaying a graphical 
interactive picture for the peripheral appliances by receiving 
a unique manipulation direction signal from each peripheral 
appliance, the display unit including: a graphical interactive 

picture structure data receipt unit for receiving the graphical 

j 

interactive picture structure data from the peripheral 

appliances to generate the graphical interactive picture; a 

< 

display's graphical interactive picture structure data storage 

j 

unit for storing the graphical interactive picture structure 

I 

i 

data received by the graphical interactive picture structure 

i 

data receipt unit; a graphical display element storage unit for 
storing a plurality of graphical display elements to generate 

the graphical interactive picture; a manipulation direction 

j 

signal receipt unit for receiving the manipulation direction 

I 

signal from the peripheral appliances; a graphical interactive 

i 

picture generation unit for receiving ; the manipulation 
direction signal received by the manipulation direction signal 
receipt unit, and for retrieving the graphical interactive 
picture structure data from the display 1 s graphical interactive 
picture structure data storage unit to generate the graphical 
interactive picture by combining the graphical display elements 
stored in the graphical display element storage unit; and a 

24 ! 
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display unit for displaying the graphical interactive picture 
generated by the graphical interactive picture generation unit. 

The graphical interactive picture generation unit may 

i 

include: a temporarily storage unit for temporarily storing 
the manipulation direction signal received by the manipulation 
direction signal receipt unit; a retrieval unit for retrieving 
the manipulation direction signal from the temporarily storage 
unit; a graphical interactive picture structure data extract 
unit for extracting the graphical interactive structure data 
from the display's graphical interactive; picture structure 
data storage unit in accordance with the manipulation direction 
signal retrieved by the retrieval unit, to generate the 
graphical interactive picture, the graphical interactive 
picture structure data including object information defining 
objects that form the graphical interactive picture, shape 
information defining shapes of the lobjects, position 
information defining positions of the! objects, action 
information defining the objects' actions that change the 

i 

graphical interactive picture based on,i the manipulation 
direction signal; a graphical display element extract unit for 
extracting bit map data from the graphical display element 
storage unit based on the shape information extracted by the 
graphical interactive picture structure datja extract unit, the 
bit map data being stored in the graphical display element 
storage unit in relation with the shape information; a 
Synthesis generation unit for generating A graphical display 

{ 
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by synthesizing the bit map data extracted by the graphical 

display element extract unit and an attribute value of a 

t 

character string attribute of each object, the attribute value 
of the character string attribute displayed on the bit map data 
being included in the object information; a placement unit for 
placing the graphical display synthesized by the synthesis 
generation unit based on the position information in the 

graphical interactive picture structure data. 

i 

The graphical interactive picture generation unit may 
further include a graphical interactive picture structure data 
update unit for updating a content of the graphical interactive 
picture structure data in the display's graphical interactive 
picture structure data storage unit based on the action 
information extracted by the graphical interactive picture 
structure data extract unit when the manipulation direction 
signal retrieved by the retrieval unit is an update direction. 

The display unit may further include- a TV receiver unit 
for converting a received program into' an image; a TV 
manipulation unit for accepting a manual manipulation to the 
TV receiver unit; a TV ! s graphical interactive picture 
structure data storage unit for storing the graphical 

interactive picture structure data related \ to the TV receiver 

i 

unit in advance; and a TV's interactive picture generation unit 
for retrieving the graphical interactive picture structure data 
from the TV's graphical interactive picture structure data 
storage unit when the TV manipulation unit receives the manual 
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manipulation to generate the graphical interactive picture by 
combining the graphical display elements, in the graphical 
display element storage unit to update the graphical 
interactive picture each time a manual manipulation is received 
by the TV manipulation unit. 

Each peripheral appliance may include: a manipulation 
, unit for accepting a user ' s action direction addressed to a 

« 

self's peripheral appliance; a user's graphical interactive 

i 

picture structure data storage unit for storing the graphical 
interactive picture structure data to display the graphical 
interactive picture as per action direction accepted by the 
manipulation unit; a control unit for controlling the self's 
peripheral appliance to set a function by giving a direction 
to transmit the graphical interactive picture structure data 
and generating a manipulation direction signal to generate the 
graphical interactive picture and a manipulation direction 
signal to updated graphical interactive picture upon acceptance 
of the action direction from the manipulation unit; a graphical 
interactive picture structure data transmission unit for 
retrieving the graphical interactive picture structure < data 
from the user's graphical interactive picture structure data 

i 

storage unit upon receipt of the transmission direction from 
the control unit, and for transmitting the retrieved graphical 
interactive picture structure data to the display unit; and 

a manipulation direction signal transmission unit for 

i 

transmitting the manipulation direction signal generated by 

\ 

i 
i 
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the control unit to the display unit. 

The control unit may include: a first signal generation 
unit for generating the manipulation direction signal as per 
action direction from the manipulation unit; and a second 
/signal generation unit for generating a predetermined 
subsidiary data signal depending on current action state of the 
self's peripheral appliance, the subsidiary data being a 
supplement of the graphical interactive picture structure data 
and constituting an integral part thereof, the subsidiary data 
signal being generated by the first signal generation unit 
together with the manipulation direction signal- 
The manipulation unit may be portable. 

The fifth object also can be fulfilled by a system 
comprising a remote controller for controlling peripheral 
appliances and a display unit for displaying a graphical 
interactive picture as per manipulation direction signal from 

the remote controller, (1) the remote controller including: 

I 

a manipulation unit for accepting an action direction addressed 

i 

to the remote controller; a control unit for controlling the 
remote controller to set a function uponj acceptance of the 
action direction from the manipulation unitj and for generating 
a manipulation direction signal to generate the graphical 
interactive picture and a manipulation direction signal to 

update the graphical interactive picture; a manipulation 

i 

direction signal transmission unit for! transmitting the 

manipulation direction signal generated by ithe control unit to 

j 
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the display unit, (2) the display unit including: an 
appliance's graphical interactive picture structure data 
storage unit for storing the graphical interactive picture 
structure data in advance to generate the graphical interactive 
picture for the peripheral appliances; a graphical display 
element storage unit for storing a plurality of graphical 
/'display elements to generate the graphical interactive picture; 
a manipulation direction signal receipt unit for receiving the 
manipulation direction signal from the remote controller; a 
graphical interactive picture generation unit for retrieving 
the graphical interactive picture structure data from the 
appliance's graphical interactive picture structure data 
storage unit upon receipt of the manipulation direction signal 
received by the manipulation direction signal receipt unit to 
generate the graphical interactive picture^ by combining the 

graphical display elements stored in the j graphical display 

i 

element storage unit based on the retrieved graphical 
interactive picture structure data; a j display unit for 

i 

displaying the graphical interactive picture generated by the 
graphical interactive picture generation unit; a TV receiver 
unit for converting a received TV program into an image; a TV 
manipulation unit for accepting a manual manipulation to the 
TV receiver unit; a TV's graphical interactive picture 

i 

structure data storage unit for storing the graphical 
interactive picture structure data as to the TV receiver unit 
in advance; a TV's graphical interactive picture generation 
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unit for retrieving the graphical interactive picture structure 
data from the TV's graphical interactive picture structure data 
storage unit when the TV manipulation unit receives the manual 
manipulation to generate the graphical interactive picture by 
5 combining the graphical display elements stored in the 
graphical display element storage unit based on the retrieved 
graphical interactive picture structure data to update the 
graphical interactive picture each time the TV manipulation 
• J unit receives a .manual manipulation. j 

lO The fifth object can be fulfilled by a system comprising 

a relay, a remote controller, and a display unit, the relay 
being a peripheral appliance for transferring an action 
direction signal from the remote controller to the display 
unit, the relay ! s function being set by thejremote controller, 

15 tfie display unit displaying a graphical interactive picture 
upon receipt of the signal from the relay, (1) the remote 

controller including: a first manipulation ; unit for accepting 

i 

an action of the remote controller; a first graphical 
interactive picture structure data storage unit for storing 

20 graphical interactive picture structure cjata in advance to 

i 

generate the graphical interactive picture as per action 

direction; a first control unit for controlling the remote 

controller to set a function upon receipt of the action 

i 

direction from the first manipulation Unit by giving a 

i 

25/ direction to transmit the graphical interactive picture 
structure data and generating a manipulation direction signal 
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tQ generate the graphical interactive; picture and a 
manipulation direction signal to update the graphical 
interactive picture; a first graphical interactive picture 
structure data transmission unit for retrieving the graphical 
interactive picture structure data from the first graphical 
interactive picture structure data storage unit upon receipt 

of the transmission direction from the first control unit to 

i 

transmit the same to the relay; and a first manipulation 

! 
f 

direction signal transmission unit for ; transmitting the 
manipulation direction signal generated by! the first control 
unit to the relay, (2) the relay including: a second 
manipulation unit for accepting an action direction addressed 
to the relay; a user's graphical interactive picture structure 

data storage unit for storing the graphical interactive picture 

j 

structure data in advance to generate the graphical interactive 
picture as per action direction addressed to the relay; a 

second control unit for controlling the relay to set a function 

i 

upon receipt of the action direction ' from the second 
manipulation unit by giving a direction to transmit the 
graphical interactive picture structure data and generating a 

i 

manipulation direction signal to generate the graphical 

i 

interactive picture and a manipulation direction signal to 

•i 

update the graphical interactive picture; a graphical 
interactive picture structure data transmission unit for 
retrieving the graphical interactive picture structure data 
from the user's graphical interactive picture structure data 
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storage unit upon receipt of the transmission direction from 
-the second control unit to transmit the same to the display 
unit; a second manipulation direction signal transmission unit 
for transmitting the manipulation direction signal generated 
5 : ! by the second control unit to the display unit; a data relay 
unit for receiving the graphical interactive picture structure 
' data from the first graphical interactive j picture structure 
data transmission unit to transmit the same to the display 
unit; and a signal relay unit for receiving the manipulation 

10 direction signal from the first manipulation direction signal 
transmission unit to transmit the same to the display unit, 
(3) the display unit including: a graphical display element 
storage unit for storing a plurality of J graphical display 
elements to generate the graphical interactive picture; a 

15 graphical interactive picture structure data receipt unit for 
receiving the graphical interactive pictur^ structure data to 
generate the graphical interactive picture j from the relay; a 

4 

\ display's graphical interactive picture structure data storage 

« • i 

unit for storing the graphical interactive picture structure 
20 data received by the graphical interactive picture structure 
data receipt unit; a manipulation direction ;signal receipt unit 
for receiving the manipulation direction signal from the relay; 
a graphical interactive picture generation unit for retrieving 
the graphical interactive picture structure data from the 
25 display's graphical interactive picture structure data storage 

i 

unit upon receipt of the manipulation direction signal received 
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by the manipulation direction signal receipt unit to generate 
the graphical interactive picture by combining the graphical 
display elements stored in the graphical display element 
storage unit based on the retrieved graphical interactive 
picture structure data, and for updating the graphical 

interactive picture each time the manipulation direction signal 

J 

receipt unit receives a manipulation direction signal; and 
a display unit for displaying the graphical interactive picture 
generated by the- graphical interactive picture generation unit. 

The fifth object also can be fulfilled by a system 
comprising a relay, a remote controller, and a display unit, 
the relay being a peripheral appliance for transferring an 

i 

action direction signal from the remote Controller to the 
display unit, the relay's function being jset by the remote 
controller, the display unit displaying a graphical interactive 
picture upon receipt of the signal from the relay, (1) the 
remote controller including: a first manipulation unit for 

accepting an action direction addressed to the remote 

I 

controller; a first control unit for controlling the remote 
controller to set a function by giving a direction to transmit 

i 

..graphical interactive picture structure data to the relay and 

generating a manipulation direction signal to generate a 

l 

graphical interactive picture and to update the graphical 
interactive picture; and a first manipulation direction signal 
transmission unit for transmitting the manipulation direction 

i 

signal generated by the first control unit to the relay, (2) 
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the relay including: a second manipulation unit for accepting 
an action direction addressed to the relay; a user's graphical 
interactive picture structure data storage unit for storing the 
graphical interactive picture structure data in advance to 
display the graphical interactive picture as per action 
direction addressed to the relay; a first graphical interactive 
picture structure data storage unit for storing in advance the 
graphical interactive picture structure data to display the 

graphical interactive picture as per graphical -inter act ive- 

i 

picture-structure data transmission direction from the remote 
controller; a second control unit for controlling the relay 
to set a function upon receipt of the action: direction from the 
second manipulation unit by giving a direction to transmit the 
graphical interactive picture structure data and generating a 
manipulation direction signal to generate the graphical 
interactive picture and to update the graphical interactive 
picture; a graphical interactive picture structure data 

! 

\ 

transmission unit for retrieving the graphical interactive 
picture structure data from the user 1 s graphical interactive 
picture structure data storage unit and the first graphical 

interactive picture structure data storage unit upon receipt 

i 

of the transmission direction from the second control unit and 
the remote controller respectively to transmit the same to the 
display unit; a second manipulation direction signal 
transmission unit for transmitting the manipulation direction 
signal generated by the second control unit to the display 
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unit; and a signal relay unit for receiving the manipulation 
direction signal received by the remote control unit to 
transmit the same to the display unit, (3) the display unit 
. including: a graphical display element storage unit for storing 
a plurality of graphical display elements to generate the 
graphical interactive picture; a graphical interactive picture 
structure data receipt unit for receiving the graphical 
interactive picture structure data to generate the graphical 

i 
! 

interactive picture from the relay; a display's graphical 
interactive picture structure data storage unit for storing the 
graphical interactive picture structure data received by the 
graphical interactive picture structure data receipt unit; a 
manipulation direction signal receipt unit for receiving the 
manipulation direction signal from the relay; a graphical 
interactive picture generation unit for retrieving the 
graphical interactive picture structure data from the display's 
graphical interactive picture structure data storage unit upon 
receipt of the manipulation direction signal received by the 
manipulation direction signal receipt unit to generate the 
graphical interactive picture by combining the graphical 

display elements stored in the graphical display elements based 

! 

on the manipulation direction, and for updating the graphical 

i 

interactive picture each time the manipulation direction signal 
receipt unit receives a manipulation direction signal; and 
a display unit for displaying the graphical interactive picture 
generated by the graphical interactive picture generation unit. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects, advantages and features of the 
'invention will become apparent from the following description 
thereof taken in conjunction with the accompanying drawings 
which illustrate specific embodiments of the; invention. In the 
drawings : 

FIG, 1 is a view depicting a structure of a conventional 
program receiver -apparatus; 

FIG. 2 is a view depicting a structure of a remote 
controller in accordance with a first embodiment of the present 
invention; ; 

FIG. 3 is a front view of a remote controller unit in 

.* 

the remote controller; 

FIG. 4 is an outer perspective view of a remote 
controller structure information supply unit furnished for the 
remote controller; j 

FIG. 5 is an outer perspective view of the remote 
controller unit and remote controller structure information 
supply unit furnished for the remote controller; 

FIG. 6 is a flowchart detailing the operation of the 
remote controller in the first embodiment; . 

FIG. 7 is a view depicting a structure of a remote 
controller in accordance with the second embodiment of the 
present invention; ■ 

FIG. 8 is a front view of a remote Controller unit in 



the remote controller in the second embodiment; 

FIG. 9 is a flowchart detailing the operation of the 
remote controller in the second embodiment; 

FIG. 10 is another flowchart detailing the operation of 
the remote controller in the second embodiment; 

FIG. 11 is a view depicting a structure of a remote 
controller in accordance with a third embodiment of the present 
invention; 

FIG. 12 is a front view of a remote ; controller unit in 

i 

the remote controller in the third embodiment; 

FIG. 13 is another front view of th6 remote controller 
unit in the remote controller in the third! embodiment; 

FIG. 14 is another front view of the remote controller 
unit in the remote controller in the third; embodiment; 

FIG. 15 is a flowchart detailing the operation of the 
remote controller in the third embodiment; 

FIG. 16 is another flowchart detailing the operation of 
the remote controller in the third embodiment; 

FIG. 17 is a front view of a modified remote controller 
unit in the third embodiment; 

FIG. 18 is a view depicting a structure of a remote 
controller in accordance with a fourth embodiment; 

FIG. 19 is a front view of a first remote controller 
unit in the remote controller in the fourth embodiment; 

FIG. 20 is a front view of a second remote controller 
unit in the remote controller in the fourth embodiment; 
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FIG. 21 is an outer perspective view of the first and 
second remote controller units in the remote controller in the 
fourth embodiment; j 

FIG- 22 is another front view of < the second remote 
controller unit in the remote controller in the fourth 
embodiment ; 

FIG. 23 is a flowchart detailing t*>e operation of the 

j 

remote controller in the fourth embodiment; 

i 

FIG. 24 .is a view depicting a structure of a remote 
controller in accordance with a fifth embodiment of the present 
invention; j 

FIG. 25 is a front view of a remote : controller unit in 

i 

! 

the remote controller in the fifth embodiment; 

FIG. 26 is another front view of the remote controller 
unit in the remote controller in the fifth 5 embodiment; 

FIG. 27 is a flowchart detailing the operation of the 
remote controller in the fifth embodiment;! 

FIG. 28 is another front view of the remote controller 

! 

unit in the remote controller in the fifth 1 embodiment; 

FIG. 28 is another front view of the remote controller 

i 

unit in the remote controller in the fifth: embodiment; 

FIG. 29 is another front view of the remote controller 
unit in the remote controller in the fifth* embodiment; 

FIG. 30 is another front view of the remote controller 

i 

! 

unit in the remote controller in the fifth embodiment; 

FIG. 31 is another front view of the remote controller 
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uipuLt in the remote controller in the fifth embodiment; 

FIG. 32 is a view depicting a structure of a menu 
synthesis apparatus in accordance with a sixth embodiment of 
the present invention; 

FIGS. 33A and 33B are' views showing examples of first 
menu information stored in a first menu storage unit in the 
sixth embodiment; ; 

FIG. 34 is an example of a menu display on TV in the 
sixth embodiment ; 

FIG. 35 is a view showing a part of synthesis rules 
stored in a synthesis rule storage unit in the sixth 
embodiment; ; 

FIG. 36 is a view showing an example of second menu 

information stored in a second menu storage unit in the sixth 

l 

embodiment; 

FIGS. 37A through 37C are views showing examples of menu 

information synthesized by a menu synthesis unit in the sixth 

i 

embodiment; i 

i 

FIG. 38 is a view showing an example of a synthesized 
menu display in the sixth embodiment; » 

FIG. 39 is a view showing an example, of a video menu in 
the sixth embodiment; 

i 

FIG. 40 is a view showing an example of menu information 
for the table of contents of a video stored; in the second menu 
storage unit in the sixth embodiment; 

j 

FIGS. 41A through 41C are views showing other examples 



qf the menu information synthesized by the menu synthesis unit 
in the sixth embodiment; 

FIG. 42 is a view showing another example of a 

i 

synthesized menu display in the sixth embodiment; 

FIGS. 43A and 43B are flowcharts detailing the operation 
of the menu synthesis apparatus in the sixth embodiment; 

FIG. 44 is a view showing a structure of a remote 
controller in accordance with a seventh ^embodiment of the 
present invention ; 

FIG. 45 is a view showing an example of the menu 

i 

information for an air conditioner in the seventh embodiment; 

FIGS. 46A through 46C are views showing examples of menu 
information synthesized by a menu synthesis unit in the seventh 
embodiment; 

FIG. 47 is a view showing an example of a synthesized 
menu display; ! 

FIG. 48 is a flowchart detailing the operation of the 

remote controller in the seventh embodiment; 

j 

FIG. 49 is a view showing an example of a display in the 

i 

seventh embodiment ; 

FIG. 50 is a view depicting a structure of a program 
transmitter and a program receiver in accordance with an eighth 
embodiment of the present invention; j 

FIG. 51 is a view explaining a panel shape 1 in the 

j 

eighth embodiment; 

FIG. 52 is a view explaining box shapes 1, 2, and 3 in 

i 
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the eighth embodiment; 

FIG. 53 is a view explaining button 1 shapes 1, 2, and 3 
In the eighth embodiment; ! 
ft FIG. 54 is a view explaining class attribute information 

'i i 

( 5, * ih the eighth embodiment; 

FIG. 55 is a view explaining panel information in the 
,/ eighth embodiment; 
• • v * FIG. 56 is a view explaining box information in the 

eighth embodiment; . 
10 FIG. 57 is a view explaining button; information in the 

eighth embodiment; 

FIG. 58 is a view explaining action information in the 
eighth embodiment; ! 

FIG. 59 is a view explaining display candidate 
15 information in the eighth embodiment; 

FIG. 60 is a view explaining shape information in the 
eighth embodiment; 

FIGS. 61A through 61C are views explaining display 
examples of an object (box) in the eighth embodiment; 
20 FIGS. 62A and 62B are views explaining display examples 

of an object (button) in the eighth embodiment; 

FIG. 63 is a view explaining an example of an 

i 

♦interactive screen in the eighth embodiment; 

FIG. 64 is a view explaining another example of the 
25 interactive screen in the eighth embodiment; 

FIG. 65 is a view explaining a remote controller in the 
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eighth embodiment; 

FIG. 66 is a view explaining how the interactive screen 
is switched in the eighth embodiment; 

FIG • 67 is another view explaining how the interactive 
screen is switched in the eighth embodiment; 

FIG. 68 is another view explaining how the interactive 
sqreen is switched in the eighth embodiment; 

FIG. 69 is another view explaining how the interactive 
screen is switched in the eighth embodiment; 

FIG. 70 is a flowchart detailing the operation of the 
program transmitter and program receiver in the eighth 
embodiment ; 

FIG. 71 is a flowchart detailing how the interactive 
screen is generated in the eighth embodiment; 

FIG. 72 is another flowchart detailing how the 
interactive screen is generated in the eight embodiment; 

FIG. 73 is another flowchart detailing how the 
interactive screen is generated in the eight embodiment; 

FIG. 74 is another flowchart detailing how the 
interactive screen is generated in the eight embodiment; 

FIG. 75 is a flowchart detailing an interactive 
"operation using the interactive screen in the eighth 
embodiment; 

FIG. 76 is a view showing a structure of a program 
transmitter and a program receiver in accordance with a ninth 
embodiment of the present invention; 
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FIG. 77 is a view explaining object information 
containing a synthesis instruction in the ninth embodiment; 

FIG. 78 is a view- explaining actinon information 
'containing a synthesis instruction in the ninth embodiment; 

t FIG. 79 is a view explaining shape information 
containing a synthesis instruction in the ninth embodiment; 
« FIG. 80 is a view explaining display candidate 

information containing a synthesis instruction in the ninth 
embodiment; 

■* FIG. 81 is a view explaining synthesized panel 

information in the ninth embodiment; 

FIG. 82 is a view explaining synthesized box information 
in the ninth embodiment; 

FIG. 83 is a view explaining the action information in 
the ninth embodiment; 

FIG. 84 is a view explaining the synthesized action 
information in the ninth embodiment; 

FIG. 85 is a view explaining the , synthesized shape 
information in the ninth embodiment; 

FIG. 86 is a view explaining the synthesized display 
candidate information in the ninth embodiment; 

FIG. 87 is a view explaining the class attribute 
information in the ninth embodiment; 

FIG. 88 is a view explaining the panel information in 
/ ; 

the ninth embodiment; 

FIG. 89 is a view explaining the text information in the 
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ninth embodiment; ; 

FIG. 90 is a view explaining the button information in 

the ninth embodiment; 

FIG. 91 is a view explaining the action information in 
the ninth embodiment; 

FIG. 92 is a view explaining the shape information in 
the ninth embodiment; 

FIG. 93 is a view explaining an example of the 
interactive screen in the ninth embodiment; 

FIG. 94 is a view exhaling synthesis instruction 
information in the ninth embodiment; 

FIG. 95 is a view explaining the panel information in 
the ninth embodiment; 

/; FIG. 96 is a view explaining the text information in the 

ninth embodiment; 

FIG. 97 is a view explaining the button information in 

the ninth embodiment; 

FIG. 98 is a view explaining the action information in 
the ninth embodiment; 

FIG. 99 is a view explaining the shape information in 
the ninth embodiment; < 

FIG. 100 is a view explaining the action information in 
the ninth embodiment; 

FIG. 101 is view explaining a synthesized interactive 
screen in the ninth embodiment; ■ 

FIG. 102 is a view explaining an interactive screen 
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switched by remote control in the ninth embodiment; 

FIG. 103 is a flowchart detailing the program 
transmitter and program receiver in the ninth embodiment; 

FIG. 104 is a flowchart detailing a synthesis operation 
for interactive screen structure specification data in the 
ninth embodiment; 

FIG . 105 is a flowchart detailing an interactive 
operation with the interactive screen in the ninth embodiment; 

FIG. 106 -is a view depicting a graphical interactive 
instruction display unit in accordance with a tenth embodiment 
of the present invention; 

FIG. 107 is a view showing a hierarchical correlation 
within classes in the tenth embodiment; « 

FIG. 108 shows a definition of the class in the tenth 
embodiment; j 

FIG. 109 shows a first definition of an action of an 
object in the tenth embodiment; 

FIG. 110 shows a second definition of the action of the 
object in the tenth embodiment; 

FIG. Ill shows a third definition of the action of the 
object in the tenth embodiment; 

FIG. 112 shows a fourth definition of the action of the 
object in the tenth embodiment; 

FIG. 113 shows a definition of a shape in the tenth 
embodiment; 

FIG. 114 shows a definition of a set value for a button 



in the tenth embodiment; 

FIG. 115 shows a definition of a candidate value in a 
set box in the tenth embodiment; ' 

v FIG. 116 shows a definition of a set value in the set 

,box in the tenth embodiment; 

FIG. 117 shows a definition of an object belonging to 
a panel class in the tenth embodiment; ; 

FIG. 118 is a view showing a structure of a graphical 
display element stored in a graphical display element storage 
unit in the tenth embodiment; 

FIG. 119 is a view showing an example of a tuner setting 
panel displayed on a graphical display unit in the tenth 
embodiment; 

FIG. 120 is another view showing an example of the tuner 
setting panel displayed on the graphical display unit in the 
tenth embodiment; ; 

FIG. 121 is another view showing an example of the tuner 
setting panel displayed on the graphical display unit in the 
tenth embodiment; 

FIG. 122 is another view showing an example of the tuner 
setting panel displayed on the graphical display unit in the 
tenth embodiment; 

FIG. 123 is a first flowchart detailing the operation 
of the graphical interactive instruction display unit in the 
tenth embodiment; 

FIG. 124 is a second flowchart detailing the operation 



of'-thfe graphical interactive instruction df splay unit in the 
tenth embodiment ; 
* FIG. 125 is a view depicting a structure of a graphical 

interactive instruction display unit in accordance with an 

eleventh embodiment of the .present invention; 

* 

:i FIG. 126 shows a 'tfirst definition of graphical 

interactive screen structure data in the eleventh embodiment; 

FIG. 127 shows a second definition of graphical 

t 

interactive screen structure data in the eleventh embodiment; 

FIG. 128 is a third definition of graphical interactive 
screen structure data in the eleventh embodiment; 

FIG. 129 is a fourth definition of graphical interactive 
screen structure data in the eleventh embodiment; 

FIG. 130 is a view showing an example of subsidiary 
data in the eleventh embodiment; 

FIG. 131 is a view showing an example of an adjustment 
panel display on a graphical display unit in the eleventh 

i 

embodiment; 

FIG. 132 is a view depicting a structure of a graphical 
interactive screen display unit in accordance with a twelfth 
embodiment of the present invention; 

FIG. 133 is a view depicting a structure of a graphical 
interactive screen display unit in a graphical interactive 
screen display direction unit in accordance with a thirteenth 
embodiment of the present invention; i 

FIG. 134 is a view showing an example of a remote- 
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controller-set-panel display on a graphical .display unit in the 
thirteenth embodiment; j 

FIG. 135 is a view depicting a structure of a modified 

I 

/graphical interactive screen instruction display unit in the 

' i 
/thirteenth embodiment; ! 

FIG. 136 is a view showing a structure of a graphical 
interactive screen display unit in a graphical interactive 
screen display direction unit in accordance with a fourteenth 
embodiment of the present invention; : 

FIG. 137 is a view depicting a structure of a modified 
graphical interactive screen display unit • in the fourteenth 
embodiment; and 

FIG. 138 is a view depicting the structure of a menu 
information synthesis apparatus in accordance with the 

i 

fifteenth embodiment of the present invention; 

:' i 

FIG- 139 is another view depicting 'the structure of a 

i 

menu information synthesis apparatus in the fifteenth 

i 

embodiment; \ 

j 

FIG. 140 is a class definition explaining the content 
of menu structure data in a TV menu structure data storage 
element and VTR menu structure data storage element in the 
fifteenth embodiment; 

FIG. 141 is a panel structure definition explaining the 
content of the menu structure data in the; TV menu structure 
data storage element in the fifteenth embodiment; 

FIG. 142 is a button structure definition explaining 
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the content of the menu structure data in thk TV menu structure 
data storage element in the fifteenth the embodiment; 

FIG* 143 is a shape definition explaining the content 

j 

dffcj' the menu structure data in the TV menu structure data 

<■ ■ 

storage element in the fifteenth embodiment; 

FIG. 144 is an action definition explaining the content 
of *• the menu structure data in the TV menu structure data 
storage element in the fifteenth the embodiment; 

FIG. 145 is a view showing the content of the element 
display data stored in a TV display element i storage element in 
the fifteenth embodiment; 

FIG. 146 is a view showing the content of synthesis 
direction data stored in a TV synthesis direction data storage 
element in the fifteenth embodiment; 

FIG. 147 is a view showing a hierarchical correlation 
between the objects 1 classes in the fifteenth embodiment; 

FIG. 148 is a panel structure definition explaining the 
content of the menu structure data in the VTR menu structure 
data storage element in the fifteenth embodiment; 

FIG. 149 is a button structure definition explaining 
the content of the menu structure data in the VTR menu 
structure data storage element in the fifteenth the embodiment; 

FIG. 150 is a shape definition explaining the content 
of the menu structure data in the VTR menu structure data 
storage element in the fifteenth the embodiment; 

FIG. 151 is an action definition explaining the content 
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of the menu structure data in the VTR menu structure data 
storage element in the fifteenth the embodiment; 

FIG. 152 is a view showing the content of the element 
display data stored in a VtJr display element storage element 
in the fifteenth embodiment; ; ; 

FIG. 153 is a view showing the content of synthesis 
.direction data stored in a VTR synthesis direction data storage 
element in the fifteenth embodiment; 

FIG. 154 .is a view showing the content of the menu 
structure data when top panels shown in FIGS. 141 and 151 are 
simply combined in the fifteenth embodiment; 

FIG. 155 is a view showing the content of the menu 
structure data when the top panels are synthesized by SIMPLE 
COMBINE in the fifteenth embodiment; 

FIG. 156 is a view showing the content of the menu 
structure data during a top-panel synthesis process by UNIQ 
COMBINE in the fifteenth embodiment; 

FIG. 157 is a view showing the content of the menu 
structure data when the top panels are synthesized by UNIQ 
COMBINE in the fifteenth embodiment; 

FIG. 158 is a view showing the menu structure data when 
set panels are simply combined in the fifteenth embodiment; 

FIG. 159 is a view showing the content of the menu 
structure data when the set panels are synthesized by SIMPLE 
COMBINE in the fifteenth embodiment; 

FIG. 160 is a view showing the content of the menu 
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structure data when the set panels are synthesized by UNIQ 
combine in the fifteenth embodiment; 

fj S FIG. 161 is a view showing the content of post- 

v 

synthesis shape data in the fifteenth embodiment; 

* *' I 

FIG, 162 is a view showing an example of a display of 

the TV menu structure data in the fifteenth embodiment; 

FIG. 163 is a view showing an example of a display of 
the VTR menu structure data in the fifteenth embodiment; 

FIG. 164 .is a view' showing an example of a display of 
a top panel after synthesizing the TV menu information and VTR 
menu information in the fifteenth embodiment; 

FIG. 165 is a view showing an emxpale of a display of 
a set panel after synthesizing the TV menu information and VTR 
menu information in the fifteenth embodiment; 

FIG. 166 is a flowchart detailing the operation related 
to the fifteenth embodiment; 

FIG. 167 is another flowchart detailing the operation 
related to the fifteenth embodiment; and \ 

FIG. 168 is another flowchart detailing the operation 
related to the fifteenth embodiment. 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fifteen embodiments of the present invention will be 
explained while referring to the drawing. 

FIRST EMBODIMENT 

FIG. 2 is a view depicting a structure of a remote 
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controller in accordance with the first embodiment. The remote 

controller includes a remote-control-structure ( RM-structure ) 

ft | 

information supply unit 201 J£nd a remote control (RM) unit 202. 
Note that appliances placed* under the control of the remote 
controller (hereinafter referred to simply as the appliances) 
are not shown in the drawing, and for the explanation's 
convenience, let the appliances be various, types of TVs made 
by a plurality of makers. The RM-structure , information supply 
unit 201 includes a RM-structure information storage unit 203, 
a RM-structure information transmission unit 204, a structure- 
information-transmission direction unit 205- The RM unit 202 
includes a structure information receipt unit 206, a structure 
information storage unit 207, a manipulation unit 208, and a 
RM signal transmission unit 209. Note that the remote 
controller referred in the first through fifth embodiments 
includes a signal (ultra-red-ray) transmitter manipulated by 
th t e user and a controller for controlling the appliances as per 
transmitted signals, and that the RM unit referred herein 
includes only the signal transmitter. 

More precisely, the RM-structure information supply unit 
201 holds the information necessary for the RM unit 202 to 
control the appliances, and transmits the information to the 
RM unit 202 via a transmission path, or namely the ultra red 
rays . 

The RM unit 202 is manipulated by the user. 

The RM-structure information storage unit 203 is a ROM 
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(Read Only Memory) and stores RM-structure information, which 
is in effect control signals transmitted from the RM unit 202 
to the appliances. In this embodiment, the control signals 
corresponding to a button group 301 (FIG. 3) on the RM unit 202 
5 fbr each type of TVs are stored, so that the RM signal 
transmission unit 209 transmits an adequate Control signal when 
^ a button is pushed. The RM-structure information transmission 
I unit 204 selectively retrieves the RM-structure information 
'from the RM-structure information storage unit 203 as per 
, 10 direction from the structure-information-transmission direction 
unit 205 to transmit the same to the RM unit 202. 

The structure-information-transmission direction unit 
205 directs the RM-structure information transmission unit 204 
to transmit the RM-structure information from the RM-structure 
15 information storage unit 203. 

The structure information receipt unit 206 receives the 
RM-structure information from the RM-structure information 
transmission unit 204 to have the structure information storage 
unit 207 store the same. \ 
20 The structure information storage unit 207 stores the 

RM-structure information received by the structure information 
receipt unit 206. 

The manipulation unit 208 is manipulated by the user to 
control the appliances. 
25 The RM signal transmission unit 209 retrieves the 

control signals, or the RM-structure information, from the 
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structure information storage unit 207 in accordance with 
manipulation signals from the manipulation unit 208. 

FIG. 3 is a front view of the RM unit 202, which is 
enclosed in a case, so that the user can easily hold it in 
hand when manipulating the same. Placed oh the front surface 
of the RM unit 202 is the button group 301, including a power 
'ON/OFF button, channel buttons, and volume buttons; "the button 
group 301 forms the manipulation unit 208. 

FIG. 4 is 'an outer perspective view of the RM-structure 
information supply unit 201, which is enclosed in a box. 
Placed on the front surface of the RM-structure information 
supply unit 201 are a toggle 401 and an ^emitter 402. The 
toggle 401 is rotated to select the type of the appliances, to 
which the RM-structure information transmission unit 204 
transmits the control signal from the RM-structure information 
storage unit 203, and it forms a part of the structure- 
information- transmission direction unit 205. The emitter 402 
emits the ultra red rays as the control signals from the RM- 
structure information storage unit 203, and it forms a part of 
the RM-structure information transmission unit 204. 

The RM-structure information is transmitted from the RM- 
structure information storage unit 203 to the RM unit 202 by 
rotating the toggle 401 while placing the RM unit 202 in an 
opposing position to the emitter 402 as shown in FIG. 5. 

The operation related to this embodiment will be 
explained while referring to the flowchart ( in FIG. 6. 
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The user places the RM unit 202 in an 1 opposing position 
to the emitter 402 of the RM-structure information supply unit 
201, and rotates the toggle 401 of the RM-structure information 
supply unit 201. Then, the structure-information-transmission 
direction unit 205 directs the RM-strupture information 
transmission unit 204 to transmit the control signal for the 
TV specified by the toggle 401 from \ the RM-structure 
information storage unit 202 (S602). Subsequently, the RM- 
structure information transmission unit 204 transmits the 
adequate control signal for the specified type of TV to the RM 
unit 202 (S604), and the structure information receipt unit 206 
receives the control signal from the RM-structure information 
transmission unit 204 (S606). Accordingly, the structure 
information storage unit 207 stores the control signal received 
by the structure information receipt unit 206 (S608). Having 
stored the control signal for the specified type of TV into the 
structure information storage unit 207, the RM unit 202 becomes 
able to control the specified type of TV. 

According to the above structure, the structure 
information storage unit 207 does not have to store the control 
signals for all kinds of appliances in advance; it can obtain 
the adequate control signal for a specified type of appliance 
when necessary* Moreover, a new type of appliance can be 
placed under the control relatively easy. 

In addition, a plurality of RM units 202 can be made for 
one RM-structure information supply unit . 201, and each can 
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receive the control signals for any desired appliances from an 

i 

: automatic information providing unit installed in the RM- 

structure information supply unit 201. 
, In this embodiment, the RM-structure information in the 

5 ' RM-structure information storage unit 203 are the control 

v *• 

signals; however, it may be, for example, the information as 
* to the appearance (color, design, etc.) or the button 
arrangement . 

Although the appliances are the different types of TVs 
10 in this embodiment, they may be the VTRs or air conditioners. 

In this case, the RM-structure information storage unit 203 
stores the control signals for these appliances, and the 
appearance of the RM unit 202 is modified accordingly. 

Further, the RM-structure information storage unit 203 
15 may store the control signals for a plurality of types of 
appliances in various kinds (TVs, VTRs, and air conditioners 
. made by a plurality of makers). 

The ultra red rays are used as the. transmission, path 
between the RM-structure information transmission unit 204 and 
20 structure information receipt unit 206; however, other 
transmission media such as radios or cables may be used as 
well. 

The RM unit 202 may additionally include a lamp to 
indicate that it has stored the RM-structure information 
25 correctly. Further, the RM unit 202 may include a means to 
notify the user when the RM-structure information does not 
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match with the button group 301 of the RM unit 202. 

i 

t ' 

SECOND EMBODIMENT 
FIG. 7 is a view depicting the structure of a remote 

I 

"controller in accordance with the second embodiment of the 
present invention. The remote controller comprises a Re- 
structure information supply unit 701 and a RM unit 702. Note 
that the RM-structure information supply unit 701 is installed 
in an appliance 703. i 

The RM-structure information supply ; unit 701 includes 
a RM-structure information storage unit 704, a RM-structure 
information transmission unit 705, and a structure-information 
request receipt unit 706. The RM unit 702 includes a structure 
information receipt unit 707, a structure information storage 
unit 708, a manipulation unit 709 , a RM signal transmission 
unit 710, and a structure-information request transmission unit 

711. 
J 

The appliance 703 includes a RM signal receipt unit 712 
and an appliance control unit 713. 

More precisely, the RM-structure information supply unit 
701 holds the information necessary for the RM unit 702 to 
control the appliance 703, and transmits the information to the 
RM unit 702 via a transmission path, or namely the ultra red 
rays. The RM unit 702 is manipulated by the user of the 
appliance 703. The appliance 703 is under the control of the 
RM unit 702. 
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The RM-structure information storage unit 704 stores RM- 
structure information, which is in effect the control signals 
transmitted from the RM unit 702 when controlling the appliance 
703. In this embodiment, the control signals for all types of 
Tys are stored, so that the RM signal transmission unit 710 
transmits an adequate control signal when a buttons of a button 
group 801 is pushed. 

The RM-structure information transmission unit 705 
transmits the RM-structure information to the RM unit 702 from 
the RM-structure information storage unit 704. 

The structure-information request receipt unit 706 
receives a RM-structure information request from the structure- 
information request transmission unit 711, and in response 
gives a direction to the RM-structure information transmission 
unit 705 to transmit the RM-structure information from the RM- 
structure information storage unit 704. The structure 
information receipt unit 707 receives ] the RM-structure 
information from the RM-structure information transmission unit 
705. The structure information storage unit 708 stores the RM- 
structure information received by the structure information 
receipt unit 707. The manipulation unit 709 is manipulated 
by the user to control the appliance 703. The RM signal 
transmission unit 710 retrieves the control signal from the 
structure information storage unit 708 in accordance with a 
manipulation signal from the manipulation unit 709, and 
transmits the same to the appliance 703. The structure- 
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information request transmission unit 711 transmits a signal 
requesting the transmission of the RM-structure information 
necessary for the RM unit 702 to control the appliance 703, 
T?he RM signal receipt unit 712 receives the 6ontrol signal from 
the RM signal transmission unit 710. The 1 appliance control 
unit 713 activates the appliance 703 as per control signal 
received by the RM signal receipt unit 712. 

FIG- 8 is a front view of the RM unit 702, which is 
enclosed in a case, so that the user can hold it easily in hand 
when manipulating the same to control the appliance 703. 
Placed on the front surface of the RM unit 702 is the button 
group 801 and a set button 802. The former includes a 
plurality of the buttons to control the appliance 703 such as 
a power ON/OFF button, channel buttons, and volume buttons, and 
it forms a part of the manipulation unit 709. The latter is 
used to transmit a request to the RM-structure information 
supply unit 701 to transmit the control signal for the 
appliance 703 to the RM unit 702, and it forms a part of the 
structure-information request transmission unit 711. 

The operation related to this embodiment will be 
explained while referring to the flowcharts in FIGS. 9 and 10. 
The. transmission paths in the form of the ultra red ray signals 
are secured between the structure-information request receipt 
unit 706 and structure-information request transmission unit 
711, the RM-structure information transmission unit 705 and 
structure information receipt unit 707, and the RM signal 



transmission unit 710 and RM signal receipt unit . 712, 
respectively. More precisely, a ultra, red ray signal 
transmitter is furnished in the RM unit 702" while a ultra red 
ray signal receiver is furnished in the appliance 703, and the 
-transmission path is formed when these two units are placed 
in an opposing position. 

To begin with, the user pushes the set button 802, then 
the structure-information request transmission unit 711 
transmits a request to the RM-structure information supply unit 
701 to transmit the control signal to the RM unit 702 (S902). 
Accordingly, the structure-information request receipt unit 706 
receives the request from the structure- information request 
transmission unit 711, and gives a direction to the RM- 
structure information transmission unit 705 to transmit the 
control signal to the RM unit 702 from the RM-structure 
information storage unit 704 (S904). Subsequently, the RM- 
structure information transmission unit 705 transmits the 
control signal to the RM unit 702 from the RM-structure 
information storage unit 704 (S906), and the structure 
information receipt unit 707 receives the control signal from 
the RM-structure information transmission unit 705 (S908). 
Accordingly, the structure information storage unit 703 stores 
the control signal received by the structure information 
receipt unit 707 (S910). Having stored the control signal in 
the structure information storage unit 708 in this way, the RM 
unit 702 becomes able to control the appliance 703. 
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Further, the appliance 703 is controlled as follows. 
To begin with, the user pushes one of the buttons of the button 
group 801, then the manipulation unit 709 gives a direction to 
the RM signal transmission unit 710 jto transmit the 
corresponding control signal (S1002). Accordingly, the RM 
signal transmission unit 710 transmits the control signal from 
the structure information storage unit 708 as per direction 
from the manipulation unit 709 to the appliance 703 (S1004), 
and the RM signal transmission unit 712 receives the control 
signal from the RM unit 702 (S1006). Then, the appliance 
control unit 713 activates and controls the appliance 703 as 
per control signal received by the RM signal reception unit 712 
(S1008) • 

According to the above structure, the structure 
information storage unit 708 does not have to store the control 
signals for all the appliances in advance, and instead, 
receives the adequate control signal for the appliance 703 
when necessary. 

Unlike the first embodiment, the RM-structure 
information supply unit 701 is installed in the appliance 703, 
and thus the user neither has to specify the appliance nor 
direct the transmission of the RM-structure information, 
further facilitating the manipulation. 

Although the appliances are various types of TVs made 
by a plurality makers in this embodiment, they may be the VTRs 
or air conditioners. In this case, : the RM-structure 
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information storage unit 704 stores the control signals for 
these appliances, and the appearance of the RM unit 702 is 
modified accordingly. 

Further, the RM unit 702 may include a button group for 
controlling the TV and VTR, and the structure information 
storage unit 708 in the RM unit 702 may accordingly store the 
control signals for both the appliances. 

The RM-structure information request is transmitted to 
the RM-structure- information supply unit 701 by the pushing of 
the set button in the RM unit 702 by the user. However, 
signals may be transmitted at regular intervals from either the 
appliance 703 or RM unit 702, and the RM-structure information 
may be automatically transmitted upon detection of the signal. 

Although ultra red ray signals are used as the 
transmission paths in this embodiment, other transmission media 
such as radios or cables are used as well. In particular, the 
RM-structure information may be transferred by means of a 
flexible disk between the RM-structure transmission unit 705 
and structure information receipt unit 707 by installing 
flexible disk drives to the RM-structure information supply 
unit 701 and RM unit 702. Also, the RM unit 702 may 
additionally include a lamp to indicate that it has stored the 
Restructure information correctly. Further, the RM unit 702 
may include a means to notify the user when the RM-structure 
information does not match with the button group 801 of the RM 
unit 702. 
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THIRD EMBODIMENT 
FIG. 11 is a view* depicting the structure of a remote 
controller in accordance with the third embodiment of the 
present invention. The remote controller comprises a RM- 
5 structure information supply unit 1101 and a RM unit 1102; the 
RM-structure information supply unit 1101 is installed in an 
appliance 1103. 

The RM-structure information supply unit 1101 includes 
a RM-structure information storage unit 1104, a RM-structure 

10 information transmission unit 1105, and a structure- 
information request receipt unit 1106. The RM 1102 includes 
a structure information receipt unit 1107, a structure 
information storage unit 1108, a manipulation unit 1109/ a RM 
signal transmission unit 1110, a structure-information request 

15 transmission unit 1111, and a manipulation display unit 1112. 

The structure information storage unit 1108 includes a RM 
signal storage unit 1113 and a display information storage unit 
1114. The appliance 1103 includes an appliance control unit 
1115 and a RM signal receipt unit 1116. 

20 The RM-structure information supply unit 1101 holds the 

information necessary for the RM unit 1102 to control the 
appliance 1103 and transmits the information to the RM unit 
1102 via a transmission path. The RM unit 1102 is manipulated 
by the user of the appliance 1103. The appliance 1103 is -under 

25 the control of the RM unit 1102. The RM-structure information 
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storage unit 1104 stores RM-structure information, which is in 
effect a control signal transmitted from the RM unit 1102 in 
controlling the appliance 1103. The RM-structure information 
transmission unit 1105 transmits the RM-structure information 
to the RM unit 1102 from the RM-structure information storage 
unit 1104. The structure- information request receipt unit 1106 
receives a RM- structure-information request from the structure- 
information request transmission unit 1111', and in response 
directs the RM-structure information transmission unit 1105 to 
transmit the RM-structure information from the RM-structure 
information storage unit 1104. The structure information 
receipt unit 1107 receives the RM-structure information from 
the RM-structure information transmission unit 1105. The 
structure information storage unit 1108 stores the RM-structure 
information received by the structure information receipt unit 
1107. The manipulation unit 1109 is used to manipulate the 
user to control the appliance 1103. The RM signal transmission 
unit 1110 retrieves the control signal from the RM signal 
storage unit 1113 as per manipulation signal from the 
manipulation unit 1109, and transmits the same to the appliance 
1103. The structure-information request transmission unit 
"llll transmits a signal requesting the RM-structure information 
'necessary for the RM unit 1102 to control the appliance 1103. 
The manipulation display unit 1112 displays figures such as 
soft buttons, or objects in a user interface screen. The soft 
buttons referred herein are not the physical buttons, but the 
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buttons actively displayed on an LC (Liquid Crystal) display 
at the bottom of an LC touch panel 1201 hy an application 
program, and serve as * the physical buttons. The RM signal 
Storage unit 1113 stores the control signal the RM unit 1102 
transmits when controlling the appliance 1103. The display 
information storage unit 1114 stores the data the manipulation 
'display unit 1112 displays. The appliance control unit 1115 
activates and controls the appliance 1103 as per control signal 
received by the -RM signal receipt unit 1116. The RM signal 
Receipt unit 1116 receives the control signal transmitted from 
the RM signal transmission unit 1110. 

FIG. 12 is a front view of the RM unit 1102. The RM 
unit 1102 is inclosed in a case, and is manipulated by the user 
to control the appliance 1103. Placed on the front surface of 
the RM 1102 are the LC touch panel 1201 and a set button 1203 
as shown in FIG. 13. The LC touch panel 1201 serves as both 
the LC display and the touch panel and displays a soft button 
group 1202 for controlling the appliance 1103 such as the TV 
or VTR. -FIG. 12 shows the LC touch panel displaying the soft 
button group 1202 for controlling the VTR, while FIG. 14 shows 
the LC touch panel 1201 displaying a soft button group 1401 
for controlling the TV. The touch panel of the LC touch panel 
1201 forms a part of the manipulation unit 1109, and the 
display of the LC touch panel 1201 forms a part of the 
manipulation display 1112. The set button 1203 is used to 
transmit a request to the RM-structure information supply unit 
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1101 in the appliance 1103 to transmit the control signal for 

the appliance 1103 and the information related to the soft 

button group 1202 to be displayed on the LC touch panel 1201. 

The set button 1203 forms a 1 parti of the structure-information 

request transmission unit 1111. Note that the RM-structure 

information storage unit 1104 stores the information related 

* 

to the soft button group 1202 in relation with the control 
signals the RM signal transmission unit 1110 transmits when a 
soft button of the soft button group 1202 is pushed. 

Next, the operation -related to this embodiment will be 
explained while referring to the flowcharts in FIGS. 15 and 16. 

Assume that "the LC touch panel 1201 displays nothing as 
shown in FIG. 13. Then, when the user pushes the set button 
1203, the structure-information request transmission unit 1111 
transmits a request to the structure-information request 
receipt unit 1106 to transmit the RM-structure information to 
the RM unit 1102 (S1502). Upon receipt of the transmission 
request from the structure-information request transmission 
unit 1111, the structure- information request receipt unit 1106 
directs the RM-structure information transmission unit 1105 to 
transmit the information related to the soft button group 1202 
and control signal from the RM-structure information storage 
unit 1104 (S1504). Subsequently, the RM-structure information 
transmission unit 1105 transmits the information related to the 
soft button group 1202 and the control signal to the RM unit 

1102 (S1506), and the structure information receipt unit 1107 
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receives the same (S1508). Accordingly, the RM signal storage 
unit 1113 stores the control signal while the display 
information storage unit 1114 stores the information related 
•fcc/the soft button group 1202 (S1510). Then, the manipulation 
display unit 1112 displays the soft button group 1202 as shown 
in FIG. 12 on the LC touch panel 1201 using the information 
stored in the display information storage unit 114 (S1512). 
In this way, the RM unit 1102 stores the control signal for 
the appliance 1103 into the RM signal storage unit 1113, and 
stores the information related to the soft button group 1202 
in the display information storage unit 1114, and becomes able 
to control the appliance 1103. Note that each soft button of 
the soft button group 1202 on the LC touch panel 1201 
corresponds to the individual control signals stored in the RM 
signal storage unit 1113. 

Next, when the user selectively pushes a soft button 
of the. soft button group 1202, the manipulation unit 1109 
directs the RM signal transmission unit 1110 to transmit the 
control signal (S1602). Thus, the RM signal transmission unit 
1110 transmits the control signal to the appliance 1103 from 
the RM signal storage unit 1113 as per direction from the 
manipulation unit 1109 (S1604), and the RM signal receipt unit 
1116 receives the control signal from the RM unit 1102 (S1606). 
Accordingly, the appliance control unit 1115 controls and 
activates the appliance as per control signal received by the 
RM signal receipt unit 1116 (S1608). 

f 67 



If -the appliance 1103 is the TV, the; remote controller 
operates in the same manner as above; however, the LC touch 
panel 1201 displays the soft button group 1401 as shown in FIG. 
14 for controlling the TV, an<i the control signals assigned to 
each soft button in the soft button group 1401 are stored in 
the RM unit 1102. 

According to the above construction, the RM unit 1102 
has to store neither the control signal for the appliance 1103 
nor the user interface such as the control buttons in advance, 
and instead, it stores the adequate control signal and user 
interface screen for a desired appliance when necessary. When 
the conventional remote controller controls a plurality kinds 
of appliances including the TV, VTR, CD player, and tuner, it 
must include a plurality of buttons: channel buttons and volume 
buttons for the TV, playback button, stop button, and record 
button for the VTR, the play button, track selection button for 
the CD player, and the tuning button for the tuner, as well as 
the control signals for each button. The number of the buttons 
further increases when one kind of appliance includes a 
plurality of types. However, the remote controller in this 
embodiment neither has to include these buttons nor store the 
control signals in advance. Moreover, the remote control can 
place a new household appliance under the control easily. 

In this embodiment, , the soft button group 1202 is 
displayed on the LC touch panel 1201. However, as shown in 
FIG. 17, a physical buttpn group and the LC display unit may 
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be combined to display the functions of each button on an LC 

I 

display 1702. j 

The RM-structure information supply unit 1101 or RM unit 

! 

1102 may include a means for specifying attribute of the RM- 
structure such as "normal", "child", "elder";. Thus, if "elder" 
is specified, larger soft buttons will appear on the LC touch 
panel 1201 compared with those appearing upon specifying 
"normal", and if "child" is specified, the soft buttons will 
appear in a simpler way on the LC touch panel 1201. 

Also, the size or position of the objects to be 
manipulated may be customized by the user with the RM remote 
controller 1102, 

FOURTH EMBODIMENT 

FIG. 18 is a view depicting the structure of a remote 
controller in accordance with the fourth embodiment of the 
present invention. The remote controller comprises a RM- 
structure information supply unit 1803 and a second RM unit 
1802. The RM-structure information supply unit 1803 is 
enclosed in a first RM unit 1801, and the appliances are not 
shown in the drawing. 

The RM-structure information supply unit 1803 includes 
a first structure information storage unit. 1804, a structure 
information transmission unit 1805, and a structure- 
information request receipt unit 1806. The second RM unit 1802 
includes a structure information receipt unit 1807, a second 
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structure information storage unit 1808, and a structure- 
information request transmission unit 1809.! 

The first RM unit 1801 controls predetermined 
/J appliances, and the second RM unit 1802 is manipulated by the 
' 5 user to control the appliances. The RM-structure information 

< • 

supply unit 1803 holds the information necessary for the second 
RM unit 1802 to control the appliances and transmits the same 
to the second RM unit 1802 via a transmission path, or namely 
u-ltra red rays. .The first structure information storage unit 

10 1804 stores RM-structure 'information. , The RM-structure 
information referred herein includes the information related 
to each object to be manipulated such as buttons furnished 
with the first RM unit 1801, and the control signals assigned 
for each object that the second RM unit 1802 ;transmits when the 

15 object is manipulated. The structure information transmission 
unit 1805 transmits the RM-structure information to the second 
RM unit 1802 from the first structure information storage unit 
1804. The structure-information request -receipt unit 1806 
receives a structure information request from the structure- 

20 information request transmission unit 1809, and in response 
gives a direction to the structure information transmission 

unit 1805 to transmit the content of the first structure 

i 

information storage unit 1804 to the second! RM unit 1802. The 
structure information receipt unit 1807 I receives the RM- 
25 structure information from the structure information 
transmission unit 1805. The second structure information 
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storage unit 1808 stores the RM-structure information received 
by* the structure information receipt unit 1807. Note that a 
manipulation means, a RM signal transmission means, and 
manipulation display means in the second RM unit 1802 are not 
shown in the drawing. 

FIG. 19 is a front view of the first RM unit 1801. 
Placed on the front surface of the first RM unit 1801 is a 
button group 1901. The button group 1901 is used to control 
the TV as an example of the appliance, and it includes a power 
button, channel buttons, and volume buttons. 

FIG. 20 is a front view of the second RM unit 1802. 
Place on the front surface of the second RM unit 1802 are an 
LC touch panel 2001 and a set button 2002. ; The LC touch panel 
2001 serves as an output unit (LC display) and also as an input 

unit (touch panel). The set button 2002 is used to transmit 

! 

a request to the first RM unit 1801 to transmit the control 

] 

signal and the information related to the • soft button group 
1901 to be displayed on the LC touch panel ;2001 to the second 

FJM unit 1802. The set button 2002 forms a part of the 

♦ i 
structure-information request transmission unit 1809. 

A soft button group 2201 as shown in FIG . 22 will be 

i 

displayed on the LC touch panel 2001 by pljacing the first RM 

i 

unit 1801 in an opposing position to the second RM unit 1802 
and pushing the set button 2002 as shown li) FIG. 21. 

Next, the operation related to this ^embodiment wi*ll be 

i 

explained while referring to the flowchart : in FIG. 23. 



When the user pushes the set button 2002 while placing 
the first RM unit 1801 in an opposing position to the second 
RM unit 1802, the RM-structure request transmission unit 1809 
transmits a request to the first RM unit 1801 to transmit the 

control siqnal and the information related to the soft button 

J 

group 2201 to be displayed on the LC touch panel 2001 to the 
second RM unit 1802 (S2302). Upon receipt of the transmission 
request, the structure-information request receipt unit 1806 
gives a direction to the structure information transmission 
unit 1805 to transmit the control signal and the information 
related to the soft button group 2201 from the structure 
information storage unit 1804 (S2304). i Accordingly, the 
.structure information transmission unit 1805 retrieves the 
control signal and the information related to the soft button 
group 2201 from the first structure information storage unit 

1804 and transmits the same to the second RMjUnit 1802 (S2306). 

i 

Then, the structure information receipt unit 1807 receives the 
control signal and the information related to the soft button 
group 2201 from the structure information j transmission unit 

1805 (S2308). Accordingly, the second structure information 
storage unit 1808 stores the control signal and information 

i 

related to the soft button group 2201 received by the structure 
information receipt unit 1807 (S2310). In this way, the second 

RM unit 1802 obtains all the functions the first RM unit 1801 

i 

has, and the soft button group 2201 as shown in FIG. 22 is 

I 

displayed on the LC touch panel 2001 . j 



According to the above structure, the second RM unit 
1802 can actively retain the same functions as the first RM 
unit 1801. For this reason, the second RM unit 1802 does not 
■have to include a plurality of buttons for all the available 
functions like a conventional self -learning remote controller, 
which can be customized in a desired manner. Further, the 
second RM unit 1802 does not have to store the control signals 
assigned for each button. 

In this -embodiment, the RM-structure information is 

transmitted from the first structure information storage unit 

i 

1804 when the structure- information request receipt unit 1806 
in the first RM unit 1801 receives the transmission request 
from the structure-information request transmission unit 1809 
in the second RM unit 1802. However, a structure-information- 

i 

transmission direction unit may be installed instead of the 
structure-information request transmission unit 1809 and 
structure-information request receipt unit 1806, and a 
structure-information-receipt direction unit may be installed 
in the second RM unit 1802, so that the structure information 

transmission unit 1805 and structure information receipt unit 

i 

1807 may be activated as per direction fromj these units. The 

j 

directions are transmitted by the pushing of a corresponding 
button on the first RM unit 1801 and the second RM unit 1802. 

i 

i 
> 

FIFTH EMBODIMENT ! 
FIG. 24 is a view depicting the structure of a remote 



controller in accordance with the fifth embodiment of the 
present invention* The remote controller comprises a Re- 
structure information supply unit 2401 and a RM unit 2402; the 
'RM-structure information supply unit 2401 is installed in an 
appliance 2403. 

The RM-structure information supply unit 2401 includes 
a RM-structure information generation unit 2404, a RM-structure 
information transmission unit 2405. The RM unit 2402 includes 
a structure information receipt unit 2406, a structure 
information storage unit 2407, a manipulation unit 2408, a RM 
signal transmission unit 2409, and a manipulation display unit 
2410. The structure information storage unit 2407 includes a 
RM signal storage unit 2411 and a .display information storage 
unit 2412. The appliance 2403 includes an! appliance control 
unit 2417 and a RM signal receipt unit 2418. 

The RM-structure information supply unit 2401 holds the 

information necessary for the RM unit 2402 to control the 

t 

appliance 2403, and transmits the same to the RM unit 2402 via 

! 

a transmission path, or namely ultra red rays. The RM unit 

2402 is manipulated by the user, and the appliance 2403 is 

! 

controlled by the RM unit 2402. 

The RM-structure information generation unit 2404 
generates the RM-structure information depending on the state 
of the appliance 2403; the RM-structure information referred 
herein is objects to be manipulated such as| buttons on the RM 
unit 2402 and the control signals assigned to each button. The 
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RM-structure information generation unit 2404 receives the 
information of the action state of the appliance 2403 (herein 
VTR) from the appliance control unit 2417. Assume that a 
recordable video tape is loaded in the VTR, then the RM- 
structure information generation unit 2404 generates the RM- 
structure information to display a soft button group 2502 as 
shown in FIG. 25 on the LC touch panel 2501 of the RM unit 
2402. If the loaded video tape is not recordable with the 

record protect tab being set, a notice of impossible recording 

i 

is transmitted from the appliance control unit 2417. In this 
case the record button and a standard/extended play button are 
eliminated and the RM-structure information is generated to 
display a soft button group 2602 as shown in FIG. 26. 

The RM-structure information transmission unit 2405 
transmits the RM-structure information generated by the RM- 
structure information generation unit 2404,! and the structure 
information receipt unit 2406 receives j the RM-structure 
information from the RM-structure informatioh transmission unit 
2405, The structure information storage unit 2407 stores the 

i 

RM-structure information received by !the RM-structure 

I 

information receipt unit 2406. The manipulation unit 2408 is 

manipulated by the user to control the appliance 2403. The RM 

i 

signal transmission unit 2409 retrieves tljie control signals 

i 

from the RM signal storage unit 2411 as per manipulation signal 

from the manipulation unit 2468, and transmits the same to the 

i 

appliance 2403. The manipulation display unit 2410 displays 

i 
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objects (figures) for the user interface such as soft buttons. 
The RM signal storage unit 2411 stores the control signal the 
RM unit 2402 transmits when controlling the appliance 2403. 
The display information storage unit 2412 stores the 
information which will be displayed by the manipulation display 
ujlit 2410 in controlling the appliance 2403. The appliance 
control unit 2417 activates and controls the appliance 2403 as 
.per control signal received by the RM signal receipt unit 2418. 
The RM signal receipt unit 2418 receives the control signal 
from the RM signal transmission unit 2409. ■ 

Next, the operation related to this embodiment will be 
explained while referring to the flowchart in FIG. 27. . 

Assume that the RM 2402 appears as shown in FIG. 25 
based on the control signal and information related to the 
soft button group 2502 from the RM structure information 
generation' unit 2404, which is identical with the RM unit 1102 
shown in FIG. 12. If the recordable video tape is removed from 
the appliance 2403 (VTR), and a non-recordbble video tape is 
loaded, the RM-structure information generation unit 2404 
generates the RM-structure information that enables 
manipulations other than the record function. The RM structure 
information transmission unit 2405 transmits the RM-structure 

> 

information thus generated by the RM-structure information 
generation unit 2404 to the RM unit 2402 (S2702). When the 
structure information receipt unit 2406 receives the RM- 

structure information from the RM-structure information 

| 
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transmission unit 2405 (S2704), the RM signal storage unit 2411 
and the display information storage unit 2412 in the structure 
information storage unit 2407 store the RM structure 
information (S2706). Accordingly, the manipulation display 
^nit 2410 displays the soft button group 2502 in accordance 
with the information in the display information storage unit 
2412 (S2708). Up to this point, the RM unit 2402 appears as 
shown in FIG. 26, deleting the record button and 
standard/extended play button. 

If a video tape written with program titles is loaded, 
the RM-structure information generation unit 2404 generates the 
soft buttons for the title of each program and a control signal 

grpup and transmits the same to the RM unit , 2402. The control 

i 

signal group includes. the control signals for fast- forwarding 
or rewinding the video tape to play a desired program. As a 
result, the RM unit 2402 appears as shown in FIG. 28, and the 
user only pushes a soft button exhibiting , the title to play 
the desired program. j 

If the appliance 2403 is the TV, an,d all the channels 
1 through 12 are available, the RM unit j2402 displays the 
twelve channels as shown in FIG. 29. When: some channels are 
not available around the midnight, only the javailable channels 
are displayed as shown in FIG. 30. if the title of the program 
are also broadcasted, it is possible to display the title 
instead of the channel number as shown in FIG. 31. 

In this way, the user can manipulate the remote 

i 
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gohtroller which suits most adequately to the current states 

r 

of the appliance. In addition, not only the LC touch panel 
2501 in the RM unit 2402 can be downsized, but also the control 
signals and the information related to the soft button groups 
2502, 2602, 2902, 3002, and 3102 are stored in a more compact 
memory . 

In this embodiment, the RM- structure information 
generation unit 2404 generates the RM-structure information for 
individual states of the appliance 2403, and the RM-structure 
information transmission unit 2405 transmits the same to the 
RM unit 2402. However, the RM-structure information generation 
unit 2404 may generate the information related to the RM- 
structure information to be deleted or added according to the 
changes in the states, and transmit the same to the RM unit 
•2402. This reduces the amount of information transmission 

between the RM-structure information supply unit 2401 and RM 

! 

unit 2402. j 

SIXTH EMBODIMENT > 

i 

FIG. 32 is a view depicting the structure of a menu 

i 

synthesis apparatus in accordance with the sixth embodiment 
of the present invention. The menu synthesis apparatus 
comprises a TV 3201 and a VTR 3202 connected to each other by 

a cable 3203, and both the TV 3201 and VTR 3202 are controlled 

i 

by a remote controller 3218. 

i 

The TV 3201 includes a first menu storage unit 3204, a 
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synthesis rule storage unit 3205, a menu synthesis unit 3206, 
a menu receipt unit 3207, a menu display unit 320Q, an 
appliance control signal transmission unit 3209, a TV's signal- 
^receipt unit 3210, a TV action unit 3211, and a RM signal 
receipt unit 3212. The TV 3201 is connected to a menu display 
unit 3213. 

The VTR 3202 includes a second menu storage unit 3214, 
a menu transmission unit 3215, a VTR's signal-receipt unit 
3216, and a VTR -action unit 3217. 

The first menu storage unit 3204 stores menu information 
as shown in FIGS . 33A and 33B. The menu information referred 
herein is composed of control information for the TV 3201 and 
information changing the screen of the control information. 

in FIG. 33A, TV is defined as the menu value of the tag 
(parenthesized with < > ) in the highest position, and the menu 

"TV" includes three sub-menus: "basic manipulation", "sub- 

i 

screen", and "adjustment". The elements cbmposing each sub- 

i 

menu are defined as <PANEL> . For example, "basic manipulation 

! 

panel" consists of six buttons such as channel-up and channel- 

I 

down buttons, and appears on the display 3213 as shown in FIG. 
34. i 

Defined in FIG. 33B are the kinds o£ buttons displayed 
: for each panel and event signals transmitted when the buttons 
are pushed. The event signals referred herein are the signals 
that activate a program managing the changes of the control 
over the appliance and the menu display. 
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The synthesis rule storage unit 3205 stores rules to 
synthesize a plurality of menu information, an example of which 
is shown in FIG. 35. 

The synthesis rules include the rules for the menu 
structure and the rules for shape selection and action. The 
rules for the shape selection are referred to when selecting 
on<=> shape for the various shaped buttons in a plurality of 
menus, The rules related to the action are referred to when the 
control is switched from one appliance to another. For 

example, when the record action is directed, the control is 

i 

shifted to the VTR 3202 from the TV 3201 before effecting the 
record action. The menu synthesis unit 3206 receives a second 
menu information from the menu receipt unit 3207 and stores 
the same • into an unillustrated self-installed buffer, and 
retrieves the first menu information and synthesis rule from 
the first menu-storage unit 3204 and the synthesis rule storage 
unit 3205 respectively to synthesize the first and second menu 
information in accordance with the synthesis; rule, and converts 

the resulting information menu into display elements, or bit 

j 

map data, transmitting the same to the menu! display unit 3208. 

For example, when the synthesis rul|e as shown in FIG. 

I 

35 is retrieved, whether 3501 is "true" or ]' false" is checked, 
and in case of "true", whether if 3502 is "true" or "false" is 
further checked. In case of "true", the process ( 1 ) is carried 
out; otherwise, 3503 is carried out. Wheh 3501 is "false", 
whether 3505 in 3504 is "true" or "false" is further checked. 



In • case of "true", the process (2) is carried out; otherwise, 
3506 is carried out. 

Upon receipt of the second menu information shown in 

i 

FIG. 36 (the second menu information will be described more in 
detail below), the menu synthesis unit 3206 retrieves the first 
menu information shown in FIGS, 33A and 33B, and carries out 
the process (1), and generates the synthesized menu information 
shown in FIGS. 37A, 37B, and 37C Note that the synthesis 
rule shown in FIG, 35 is a part of the synthesis rules and 
only regulates the synthesis of the , highest-position 
information sandwiched by <MENU> and </MENU> . 

The menu receipt unit 3207 receives the second menu 
information from the menu transmission unit 3215 and transmits 
the same to the menu synthesis unit 3206. 

! 

The menu display unit 3208 receives tt>e display elements 

! 

in the form of the bit map data from the menu synthesis unit 

3206 and displays the same on the display 3213 . 

i 

For example, the synthesized menu information in FIGS. 
37A, 37B, and 37C appears on the display 3213 as shown in FIG. 

38. Thus, when the user selects the TV unit 3201 using the 

I 

remote controller 3218, a basic manipulation! panel 3801 appears 
on the display 3213, and a video playback panel 3802 appears 
on the display 3213 when the user selects the VTR 3202. 

The appliance control signal transmission unit 3209 

i 
i 

receives the RM signal from the RM signal receipt unit 3212, 
and then converts the same into appliance control signal. In 



case of the appliance control signal for the TV 3201, the 
appliance control signal is transmitted to the TV action unit 
3211 by means of the TV r s signal receipt unit 3210. In case 
of the appliance control signal for changing the menu screen, 
the appliance control signal is transmitted to the menu 
synthesis unit 3206. In case of the appliance control signal 
/for the VTR 3202, the appliance control signal is transmitted 
to the VTR's signal receipt unit 3210. 

The TV's" signal receipt unit 3210 transmits the 
appliance control signal from the appliance control signal 
transmission unit 3209 to the TV action unit 3211, 

The TV action unit 3211 controls the TV 3201 as per 

i 

appliance control signal from the TV's signal receipt unit 
3210. 

The RM signal receipt unit 3212 receives the RM signal 
from the remote control unit 3218 to transmit the same to the 
appliance control signal transmission unit i3209 . 

The second menu storage unit 3214 stores the control 
information for the VTR 3202 and the second menu information. 
The second menu information referred herein Its the information 
used to change the screen based on the control information, 
and example of which is shown in FIG. 36. In the drawing, VTR 
is defined as the highest-position menu value. The "playback 
panel" indicated by the sub-menu of the second menu in FIG. 36 
appears on the display 3213 as shown in FIG. 39. 

Assume that the menu information for generating the 
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table of contents is additionally stored in the second menu 
storage unit 3214 as the second menu information, and that the 
menu synthesis unit 3206 holds the menu information shown in 
FIGS. 37A, 37B, and 37C. Then, the menu synthesis unit 3206 
* synthesizes the additional menu information and the already- 
synthesized menu information, and the.: resulting menu 
information appears as FIGS . 41A, 41B, and 41C. Note that the 
menu synthesis unit 3206 carries out the process (2) of the 
synthesis rule in FIG. 35. As a result, the display 3213 shows 
updated menu information as shown in FIQ. 42. Thus, by 
selecting VTR, and further selecting "make table of contents" 
(sub-menu) using the remote controller 3218, a table-of- 
content creation menu 4201 appears on the display 3213. 

The menu transmission unit 3215 detects the connection 
between the TV 3201 and menu receipt unit 3207 via the cable 

3203, and in response retrieves the second! menu information 

i 

from the second menu storage unit 3214 to transmit the same 

i 

to the menu receipt unit 3207. J 

The VTR's signal receipt unit 3216 receives the 

i 

appliance control signal from the appliance control signal 

} ! 

transmission unit 3209, and transmits the! same to the VTR 
* 

-action unit 3217. I 

The VTR action unit 3217 controls the VTR 3202 as per 
appliance control signal from the VTR's signal receipt unit 
3216. j 

The remote controller 3218 transmits' the RM signal to 
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the TV 3201. 

I Next, the operation related to this embodiment will be 
explained while referring to the flowcharts 1 in FIGS. 43A and 
43B. 

When the menu transmission unit 3215 in the VTR 3202 
detects the connection between the VTR 3202 and TV 3201 
(S4302), it retrieves the second menu information from the 
second menu information storage unit 3214 (S4304), and 
transmits the same to the TV's menu receipt unit 3207 in the 
TV 3201 (S4306). Upon receipt of the second menu information 

from the menu transmission unit 3215, the TV's menu receipt 

i 

unit 3207 transmits the same to the menu synthesis unit 3206 
(S4308). 

I 
I 

Accordingly, the menu synthesis unit 3206 retrieves the 
first menu information from the first menu ^storage unit 3204 

! 

(S4310), and further retrieves the synthesis rule from the 
menu synthesis rule 3206 to synthesize the' first and second 
menu information under the synthesis rule (S4312). The menu 

display information 3208 displays the resulting synthesized 

I 

menu information on the display 3213 (S4314|). 

As the remote controller 3218 transmits the RM signal 

to the TV 3201 (S4316), the RM signal receipt unit 3212 

receives the RM signal (S4318). Then, the .appliance control 

signal transmission unit 3209 converts the received RM signal 

into the appliance control signal (S4320). 
*• 

Further, the appliance control signal transmission unit 
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3209 judges whether the appliance control signal is the control 
sighal to active the VTR 3202 (S4322). If so, the appliance 
control signal transmission unit 3209 transmits the same to the 
VTR's signal receipt unit 3216, and the VTR 1 s signal receipt 
unit 3216 receives the same (S4324). Accordingly, the VTR 
action unit 3217 controls the VTR 3202 as per appliance control 
signal (S4326). 

If the appliance control signal is not the control 
signal to activate the VTR 3202 in S4322, then the appliance 
control signal transmission unit 3209 judges whether the 
appliance control signal is the control signal to activate the 
TV 3201 .(S4328). if so, the appliance control signal 
transmission unit 3209 transmits the same to j the TV action unit 
3211, which controls the TV 3201 as per J appliance control 
signal (S4330). If the appliance control .signal is not the 

control signal for the TV 3201 either, the' appliance control 

\ 

signal transmission unit 3209 transmits the jcontrol signal for 

! 

.1 

directing the change of the menu display to ;the menu synthesis 
unit 3206, which accordingly controls the jmenu display unit 
3?08 (S4332). 1 

As has been stated, when the TV and! VTR are connected 
to each other, it has become possible to manipulate both 
appliances using a single user interface that has synthesized 
the user interfaces for the TV and VTR. 

The TV and VTR are used as the appliances in this 
embodiment* However, the appliances may be the audio 
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apparatus, air conditioner, electric fan, washers, or microwave 
oven, and these appliances may be controlled by a single user 
interface as well* 

f 

SEVENTH EMBODIMENT 

FIG. 44 is a view depicting the structure of a remote 
controller in accordance with the seventh embodiment of the 
present invention. The remote controller employs the menu 
synthesis apparatus of the sixth embodiment, and comprises a 
menu information demand unit 4401, a menu receipt unit 402, a 
menu synthesis unit 4403 , a menu display unit 4404, and an 
appliance Control signal transmission unit 4405. 

The menu information demand unit j 4401 transmits a 
signal, or namely ultra red rays, that demands the transmission 
of the menu information when the remote controller is placed 
in an opposing position to the external appliances (not shown), 
*such as the TV, VTR, and air conditioner, j 

When the external appliances receives; the demand signal, 
they transmit the menu information to the remote controller. 
For example, the TV transmits the menu information shown in 
FIGS. 33A and 33B, and the VTR and air conditioner transmit the 
menu information shown in FIGS. 36 and 45, respectively. 

The menu receipt unit 4402 receives the menu information 
from the external appliances and transmits the same to the menu 
synthesis unit 4403. i 

The menu synthesis unit 4403 includes the synthesis 

86 



rule storage unit 3205 of the six embodiment, and stores the 
synthesis rule shown in FIG . 35. Upon receipt of the- menu 
information from a plurality of the external appliances by way 
.-of the f menu receipt unit 4402, the menu synthesis unit 4403 
synthesizes the plurality of the menu information as per 
synthesis rule. When the menu information is for the TV, VTR, 
and air conditioner, then the resulting , synthesized menu 
information are shown in FIGS. 46A, 46B, and 46C, respectively. 
The synthesized menu information is converted into the bit map 
data and transmitted to the menu display unit 4404. 

The menu display unit 4404 comprises ; an LC panel and a 

i 

touch panel; the latter is layered on the former. The menu 
display unit 4404 displays the menu information from the menu 
synthesis unit 4403 as the highest-position menu 4701 shown in 

i 

FIG. 47. If the user selects "TV" in the ; highest-position, 
then the menu display unit 4404 displays a basic manipulation 

i 

panel 4702 for the TV; if the user selects "VTR", then it 

displays a playback panel 4703, and if the ; user selects "air 

i 

$ conditioner" , then it displays a set panel 4^04. When the user 
pushes a power button 4705 while the set pdnel 4704 is being 
displayed, the manipulation signal for the air conditioners 

power is transmitted to the appliance control signal 

i 

transmission unit 4405. 

The appliance control signal transmission unit 4405 
receives the manipulation signal from the menu display unit 
4404, and transmits the same to the external appliances. When 
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the manipulation signal is the one to manipulate the air 
conditioner's power, then the appliance control signal 
transmission unit 4405 transmits the control signal of the 
power to the air conditioner, so that the power button is 
switched from ON to OFF or vise versa. 

Next, the operation related to this embodiment will be 
explained while referring to the flowchart in FIG. 48. 

The menu information demand unit 4401 demands the 
transmission of* the menu information from the external 
appliances (S4802), and more than one ekternal appliance 
receives the demand signal (S4804). Accordingly, the external 
appliances transmit the menu information ip response to the 

! 

demand signal (S4806), which is received by the menu receipt 
unit 4402 (S4808). Then, the menu synthesis unit 4403 

synthesizes the menu information from more! than one external 

i 

appliance (S4810), and the menu display unit; 4404 displays the 
Resulting synthesized menu inf ormation( S4812 ) . 

/j In this embodiment, the demand signal is transmitted 

' - ' s i 

from the menu information demand unit 4401 when the remote 
controller is placed in an opposing position to the external 
appliances. However, the demand signal may be transmitted only 
to the available external appliances with ! their respective 
switches being turned on, or only to the external appliances 
installed within the same room where the remote controller is 
placed, in other words, within the reach of the RM signal. 

As has been explained, the manipulation menus for a 
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plurality of appliances can be actively synthesized into a 
signal user interface, thus serving as the remote controller. 

Although the types of the controllable appliances are 
displayed on the menu display unit 4404 as the highest- 
position menu, the remote controller may detect the location 
of the individual appliances and displays the layout thereof 
within a room as shown in FIG. 49 as the highest-position menu. 
In this case, a sensor for detecting the location of each 
appliance is additionally furnished, and each appliance 
transmits a signal indicating their respective locations. The 
sensor receives the signals from the external appliances and 
detects the location of each external appliance. Then, the 

menu synthesis unit 4403 retrieves the data related to the 

i 

location of each appliance and displays the same by the menu 

i 

display unit 4404. j 

j 
j 

EIGHTH EMBODIMENT I 

FIG. 50 is a view depicting the structure of a program 
receiver and a program transmitter in accordance with the 
eighth embodiment of the present inventions 

A program transmitter 5001 includes a program 
transmission unit 5010 and a information receipt unit 5019. 

The program transmission unit 5010 multiplexes an 
information supply program and the data specifying the 
structure of interactive screen ( hereinafter referred as to the 
IS structure specification data) and transmits the resulting 
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multiplex data. The data are multiplexed by, for example, a 
method multiplexing image and text information in the text 
broadcast system, or a method for transmitting the signal and 
video separately via a sound B mode in the satellite broadcast. 
The journal published by Image Electronic Meeting, Volume 12, 

r' 

No. 4 (1983), pp. 318-326 ' describes the former method in 
detail. 

The program receiver 5002 includes a signal receipt unit 
5011, a signal separation unit 5012, a display object storage 
unit 5013, an interactive screen (IS) generation unit 5014, a 
display unit 5015, a RM signal receipt unit 5016, an 
information transmission unit 5017, and a basic action object 
storage unit 5018. 

The signal receipt unit 5011 receives either the 
# multiplex signal of the information supply program and IS 
structure specification data or the information supply program 
' ^ignal from the program transmission unit 5010 of the program 

* ! 

; 

transmitter 5001, and transmits the same to the signal 

\ 

separation unit 5012. 

The signal separation unit 5012 judges whether the 

signal from the signal receipt unit 5011 is a multiplex signal 

or not. In case of the multiplex signal, the signal separation 

unit 5012 separates the received signal into two signals; the 

information supply program signal and IS structure 

♦ 

specification data signal. The former is transmitted to the 
display unit 5015 and the latter to the IS generation" unit 
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5014. In case of non-multiplex signal, the signal separation 
unit 5012 passes the received signal to the display unit 5015. 
If the multiplex signal does not include the IS structure 
specification data signal as ■ is with the text broadcast, the 
signal separation unit 5012 passes the received signal to the 
display unit 5015. 

The display object storage unit 5013 stores file names 
in relation with the bit map data; the file names are used to 
discriminate the' objects, or the basic elements composing the 
interactive screen, and the bit map data represent the shape 
of each object. | 

The file name "panel shape 1" represents the frame of 
a rectangle 5101 shown in FIG. 51. The rectangle 5101 appears 
as the window on the display unit 5015. ! 

The file names "box shape 1", "box shape 2", and "box 
shape 3" represent the figures 5201, 5202, 5203 shown in FIG. 
52, respectively. : 

Two of the figures 5201 through |5203 fit in the 
rectangle 5101 horizontally, and three of j them fit therein 
vertically. The figure 5202 representing ; "box shape 2" is 
shadowed in the drawing; however, the figure, 5201 representing 
"box shape 1" may be high-lighted on the display unit 5015. 
The figure 5203 representing "box shape 3" is shadowed at the 
three sides, but the figure 5201 representing "box shape 1" is 
displayed in 3D (three-dimension) on the display unit 5015. 

The file names "button shape 1", "button shape 2", and 
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"button shape 3" correspond to the figures 5301, 5302, and 5303 
shown in FIG. 53, respectively. The figures 5301, 5302, and 
, 5303 fit in the rectangle 5101 vertically. The figure 5301 
representing "button shape 1" is shadowed. However, the figure 
530 representing "button shape, 2" may be high-lighted on the 
display unit 5015. Likewise, the figure 5303 representing 
"button shape 3" is shadowed at the three sides, but the figure 
5301 is displayed in 3D on the display unit 5015. 

The IS generation unit 5014 includes a first receipt 
unit and a second receipt unit: the first receipt unit receives 
the IS structure specification data signal from the signal 
separation unit 5012; the second receipt unit receives the 
signal from the RM signal receipt unit 5016. Further, the IS 
generation unit 5014 includes a process unit for interpreting 
the signals received by the first and second; receipt units for 
further processing, and a storage unit, as ; well as a display 
control unit for having the display unit ! 5015 display the 
.interactive screen image. 

The IS structure specification data received by the 
first receipt unit comprise following definitions: 

1) a class definition including class attribute 
information 

2) a panel definition including panel information 

3 ) a box definition including box information 

4) a button definition including button information 

5) action definition including action information 
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6) a display candidate definition including display 
candidate information 

7) a shape definition including shape information, 

■i * 

v • The class attribute information shown in FIG. 54, are 

5 ? the information that defines attribute for each class (group) 
* 

of the objects forming the interactive screen. For example, 
' the attribute of the object belonging to a specific class is 

defined by " < I ELEMENT • • • > 

For example, "<! element box(character__string & shape 
•10 ••• & STATE_CANDIDATE> " defines that the object belonging to 
: ' the "box" class has the attribute, "character string", "shape", 

•••"state candidate". Similarly, the object belonging to the 
"button" class has the attribute, "character string", "shape", 
"action", "selection state", and "focus state"; the object 
15 belonging to the "panel" class has the attribute, "shape". 

The panel information is the information that defines 
the objects belonging to the panel class as ; shown in FIG. 55. 
The panel information includes panel attribute information 
defining the attribute of the panel, the name of the box 
20 specifying the objects belonging to the box class within the 
panel, the name of the button specifying the objects belonging 
to the button class within the panel, and position information 
specifying the abstract position of the objects specified by 
the box or the button name. 
25 "<PANEL SHAPE « PANEL_SHAPE>QUIZ 1" defines an object 

<QUIZ> belonging to the panel class. "SHAPE « PANEL_SHAPE" 
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means that the object has "panel shape" as its shape attribute. 
Objects between " < PANEL •••>" and " </PANEL> are the objects 
belonging to the "quiz 1 panel". That is to say, there exist 
the objects, or "America box", "France box", "England box", 
"Germany box", and "transmission box", in the "quiz 1 panel". 
In addition, "<v>", "</v>", "<h>", "</h>" define the position 
of the objects. If the objects are sandwiched by "<v>" and 
"</v>" , they are placed vertically, and if sandwiched by "<h>" 
and "</h>", then horizontally. Thus, the two ( objects, "America 
box" and "France box", are placed horizontally. 

The box information are the information that defines the 
objects belonging to the box class as shown in FIG. 56. 

In "<BOX ••♦>XXX", "..•" defines the attribute of the 
object XXX in the box. For example, followings are the 
attribute of the object in "America box": 

the character string to be displayed is "America" 
the shape is "box shape" 
the action is "America box action" 
the selected state at the initialization is "OFF" 
the focus is "ON" at the initialization 
the state candidate is "0NJ3FF candidate". 
"The selection state is ! 0N tn means that the object is 
selected, and "the focus is ! 0N T " means that the object is 
subject to manipulation, and thus displayed by being high- 
lighted or shadowed. 

The button information is the information that defines 
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the objects belonging to the button class as shown in FIG. 57. 

Like the box objects, " • • • " in " < BUTTON* • -XXX" defines 
the attribute of the object XXX in the button. For example, 
fallowings are the attribute of the object in the "transmission 
button" : 

'/ the character string to be displayed is "transmission" 

the shape is "button shape" 

the action is "transmission button action 11 
the focus is "OFF" at the initialization. 

The action information is the information that defines 

j 

the action of the objects belonging to the box class and button 
class as shown in FIG. 58. s 

In " <ACTION>YYY" , YYY represents the action name.. In 
" < EVENT > AAA" , AAA represents the event name, and in 
" < COMMAND >BBB, BBB represents the command riame executed when 
the event specified by the preceding event name occurs. In 
"< ARGUMENT >CCC, CCC represents the argument delivered to the 
command specified by the preceding command name. For example, 
in the "America box action", when an event "right" is 
transmitted, a command "shift focus" is executed. Also, in 
executing the "shift focus", an argument! "France box" is 
delivered. In short, when the event "right" is transmitted to 
the "America box", the focus is shifted to the "France box", 
and the "France box" has become the subject of manipulation. 

The display candidate information is the information 
that defines a plurality of value candidates as shown in FIG. 



59. 

In "<CANDIDATE DATA>AAA" , AAA represents the candidate 
data name, and indicating that it has the value sandwiched 
between " <CANDIDATE_DATA>AAA" and " </CANDIDATE_DATA> " as the 
candidate value. For -example, " <candidate_data>ON_OFF_ 
CANDIDATE" defines the candidate value of "ON_OFF candidate": 
the "ON_OFF candidate" includes the candidate values of "ON" 

i 

and "OFF". 

The shape' information is the information that defines 
the shape data as shown in FIG. 60. 

The data defined between 11 < SHAPE > 11 and </SHAPE> are the 
definition of the shape data, and the shape data vary according 
to the conditions given by the definition of " <CONDITION> " . 
For example, the "box shape* 1 appears as the "box shape 1" when 
"FOCUS_STATE = OFF & SELECTIONS TATE = OFF 11 is given, while 
appearing as the "box shape 2" when "F0CUS_STATE = ON & 
SELECT I0N__STATE = OFF" is given, and as the '.'box shape 3" when 
"FOCUS = ON" is given. 

The first receipt unit in the IS generation unit 5014 
receives the IS structure specification • data signal and 
transmits the same to the process unit. 

The process unit decodes the IS structure specification 
data signal and has the storage unit store the same. Further 
it retrieves the panel information (FIG. 55) from the stored 
•IS structure specification data, and judges whether it is the 
panel attribute information, box name, button name, or start 



position information, or end position information per unit (per 

line in FIG. 55). 

In case of the panel attribute information, the process 

unit obtains the attribute value of the panel "panel shape", 

arfd retrieves the shape information (FIG. 60) from the storage 

unit using the "panel shape". Then, it turns out that the 
''panel shape" is "panel shape .1", and the process unit obtains 

the bit map data for the "panel shape 1" (FIG. 51) from the 

display object storage unit 5013, transmitting the same to the 

display control unit. 

In case of the box name, the process unit stores the 

box name in a panel information buffer in the storage unit, and 
retrieves the corresponding box information (FIG. 56) from the 
storage unit. Further, the process unit obtains the attribute 
values for the "focus state" and "shape" from the box 
information, and retrieves the shape information (FIG. 60) from 
the storage unit to obtain the attribute value of the shape 

•"box shape". The process unit obtains one of the "box shape 
1", "box shape 2", and "box shape 3", which is in compliance 
with the condition of the "box shape", as the file name, and 
the obtains adequate bit map data for the file name from the 
display object storage unit 5013. Also, the process unit 
extracts the attribute value of the "character string" from the 
box information to synthesize the same with the bit map data 
to. generate the box figure, transmitting , the same to the 
display control unit. 

97 



For example, if the attribute value of the "character 
string" is "America", then the object in the "America box" 
appears as shown in FIGS. 61A, 61B, and 61C under the 
fCOnditions "FOCUSJSTATE = OFF & SELECTIONIST ATE = OFF" , "FOCUS_ 
STATE = ON & SELECT 1 0N_J3 TATE - OFF", and "SELECTIONIST ATE = 
ON" , respectively . 

In case of the button name, the process unit stores the 
button name in the panel information buffer in the storage 
unit, and retrieves the corresponding button information (FIG. 
57) from the storage unit. The process unit further obtains 
the attribute value of the "focus state" in the button 
information to retrieve the shape information (FIG. 60) from 
the storage unit, obtaining the attribute value of the "shape" 
and "button shape". Further, the process unit obtains one of 
the "button shape 1" , "button shape 2", and' "button shape 3", 
which is compliance with the condition of the "button shape", 
as' the file name, and retrieves the bit map data corresponding 
to the file name from the display object storage unit 5013. 
The process unit extracts the attribute value "transmission" 
of the "character string" in the button information to 
synthesize the same with the bit map data to generate the 
button figure, transmitting the same to the display control 
unit. 

For example, if the attribute value of the "character 
string" is the "transmission", then the object in the 
"transmission button" appears as shown in FIGS. 62A and 62B 
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when the condition is "FOCUSJSTATE = OFF" and "FOCUS_STATE - 
ON" , respectively . 

In case of the start position information, the process 
junit stores the start position information " <v>" or "<h>" in 
the panel information buffer.; 

In case of the end position information, the process 
unit checks whether the end position information "</v>" or 
"</h>" is consistent with the start position information 
previously stored in the panel information buffer. When the 
start position and end position information = are inconsistent, 
or the IS structure specification data are not the end position 
information, the process unit transmits an error message to the 
display control unit. , ! 

When the end position information is in consistent with 
the start position information, the process unit retrieves the 
object names sandwiched between the position information in the 
panel information buffer in the f irst~in-f irst-out order, and 
places the same virtually as per position information, 
transmitting the same to the display control unit. While at 
the same time, the process unit generates a name for a set of 
the objects, and deletes the names of the virtually placed 
objects and the position data from the panel buffer and stores 
the set name instead. 

The process unit checks whether the processing for one 
panel information is completed or not, and transmits a notice 
of display to the display control unit when the processing is 
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completed . 

The process unit in the IS generation unit 5014 receives 
San event signal from the second receipt unit, and in response 

it 

.bhanges the interactive screen as per user 1 s instruction. Upon 

receipt of the event signal, the process unit retrieves the box 

information (FIG. 56) or button information (FIG. 57) from the 

storage unit, and extracts the object subject to manipulation 

(the object exhibiting " F0CUS_STATE - ON") and its action name. 

Then, the process unit retrieves the action information (FIG. 

58) from the storage unit based on the retrieved action name 

to retrieve the command name and argument j for the received 

event name. Further, the process unit retrieves the content 

of the basic action corresponding to the command name from the 

basic action object storage unit 5018, and accordingly rewrites 

the box information (FIG. 56)' or button information (FIG. 57) 

in the storage unit, and 1 changes the interactive screen, 

transmitting the notice of display to the display control unit. 

If the command name is "attribute value transmission", the 

process unit retrieves the attribute value of the selection 

state candidates of each box specified by the "argument", and 

transmits the same to the information transmission unit 5017. 

The display control unit receives the bit map data of 

the panel shape, box figure, button figure, and virtually 
f 

placed objects as well as display notice from the process unit, 
and has the display unit 5015 display the interactive screen 
on the window. FIG. 63 is an example of the interactive 
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screen. 

Also, the display control unit receives the display 
notice for the interactive screen changed as per user's 
^instruction, and has the display unit 5015 display the changed 
* interactive screen. 

The storage unit includes the panel information buffer 
and stores the IS structure specification data from the process 
unit; the panel information includes data related to the 
objects therein and the data related to the placement thereof. 

The display unit 5015 includes a screen (CRT), and a 
control unit for having the screen display the information 
supply program as per information supply program signal from 
the signal separation unit 5012. As shown in FIG. 64, the 
display unit 5015 displays the interactive' screen under the 
control of the IS generation unit 5014 by establishing a window 
6402 in a screen 6401. 

The interactive screen in FIG. 63 is changed as shown 
in FIGS. 66 through 69 as the user pushes the (right), 
"J," (down)", "selection", " ]/ (down) buttons successively 
on a remote controller 6501 shown in FIG. 65. 

The RM signal receipt unit 5016 receives the signal 
from the user by means of the remote controller 6501, and 
transmits the same to the IS generation unit 5014. 

The remote controller 6501 includes a power button 6502, 
an up button 6503 "f" , a right button 6504 "->", a down button 
"J,", a left button 6506 "<-", a "selection" button 6507, and a 
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"determination" button 6508. When the user pushes the power 
button 6502 once, the RM signal receipt unit 5016 becomes 
ready, and when he pushes the power button 6502 again, the 
switch is turned off- When the user pushes the "up" button 
6503, an "up" event signal is transmitted to the RM signal 
receipt unit 5016. Likewise, when the user pushes the "right" 
button 6504, a "right" event signal is transmitted to the RM 
signal receipt unit 5016; when the user pushes the "down" 
button 6505, "left" button 6506, "selection" button 6507, and 
"determination" button 6508, a "down" event signal, a "left" 
event signal, a "selection" event signal, and a "determination" 
event signal are transmitted to the RM signal receipt unit 
5016 , respectively. 

The information transmission unit 5017 transmits 
interactive screen (IS) data (each attribute value for the 
selection state candidate) transmitted from the IS generation 
unit 5014 to the information receipt unit 5019 of the program 
transmitter 5001. 

The basic action object storage unit 5018 stores the 
following basic actions specified the "command": "shift focus", 
"switch state", and "transmit attribute value". The "shift 
focus" basic action means to switch the focus of the object 
subject to manipulation from "ON" to "OFF", ,and to switch the 
focus of the object specified by the "argument" from "OFF" to 
"ON". The "switch state" basic action means to switch the 
.selection state of the object subject to manipulation from 
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"OFF" to "ON". The "transmit .attribute value" basic operation 
means to retrieve all the attribute values "ON" or "OFF" of the 
selection state candidates of the object specified by the 
argument . 

The operation related to this embodiment will be 
explained while referring to the flowcharts in FIGS. 70 through 
75. 

The program receiver 5002 receives the information 
supply program signal from the program transmitter 5001 
(67002). The signal separation unit 5012 checks whether the 
received signal is a multiplex signal or not (S7004), and in 
case of a multiplex signal, it separates the received signal 
into the information supply program signal and IS structure 
specification data signal (S7006). The process unit checks 
whether the received signal is the information supply program 
signal or not (S7008). If it is not the information supply 
program signal, the process unit checks whether there is the 
'IS structure specification data signal or not (S7010). If the 
VIS structure specification data signal exists, the signal 
' separation unit 5012 transmits the same to the IS generation 
unit 5014 (S7012). 

The IS generation unit 5014 generates the interactive 
screen using the objects stored in the display object storage 
unit 5013 based on the IS structure specification data signal 
(S7014) . 

If the signal is not the multiplex signal in S7004 and 
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judged as to be the information supply program signal in S7008, 
then the signal separation unit 5012 transmits the information 
•supply program signal to the display unit 5015 when there 
exists the IS structure specification data signal in S7010 

. y . ■ * 

I 5 (S7016). The display unit 5015 thus generates the screen for 

' ■ $ 

/ v? the information supply program (S7018). 

' , The interactive screen generated in S7014 appears 

together with the information supply program on the window 6402 
=j of the display , unit 5015 (S7020), 

EST 

;J 10 If the program receiver 5002 stops the signal reception 

(S7022), the display unit 5015 terminates the display 

II operation; otherwise it return to S7002. 

Next, S7014, where the interactive screen is generated 
f by the IS generation unit 5014, will be described more in 

? 15 detail, 

* f The process unit in the IS generation unit 5014 

retrieves the panel information in the IS structure 

specification data from the storage unit (S7102), and retrieves 

one unit of information from the panel information (panel 

20 attribute information, box name, button name, start position 

information, and end position information). Then, the process 

unit judges whether all the data related to the panel 

• information are processed or not, and if not (S7106), the 

process unit further checks whether the retrieved panel 

25 information per unit is the panel attribute information or not 

(S7108). In case of the panel attribute information, the 
t 
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process unit retrieves the "shape" attribute value from the 
panel attribute information (S7110); otherwise it proceeds to 
S7116. Subsequently, the process unit retrieves the shape 
information (file name) that matches with the "shape" attribute 
value retrieved in S7110 , obtaining the shape data specified 
by the shape information (S7112). The shape data are the bit 
map data herein (S7114). Then, the process unit returns to 
S7104 and proceeds to S7108,/ and checks whether the retrieved 
information per unit is the box name or not(S7116). In case 
of the box name, the process unit proceeds to S7202; otherwise 
it further checks whether the retrieved information per unit 
is the button name or not (S7118). In case of the button name, 
the process unit proceeds to S7302; otherwise, it further 
checks whether the retrieved information per unit is the start 
position information or not (S7120). In case of the start 
position information, it stores the same in the panel 
information buffer of the storage unit (S7122); otherwise, it 
further checks whether the retrieved information per unit is 
the end position information or not (S7124). In case of the 
end position information, the process unit checks whether the 
end position information is in consistent with the ^ most 
recently stored start position information in the panel 
information buffer (S7126). If there is consistency, the 
process unit proceeds to S7402. If the retrieved data is not 
the end position information in S7124, or there is no 
* consistency in S7126, then the process unit transmits the error 
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message representing "incorrect; definition for panel 
information" to the display control unit (S7128), and ends the 
•generation of the interactive screen. 

The process unit additionally stores the box name in the 
, panel information buffer, and retrieves the box information 
•corresponding to the box name (S7204), further obtaining the 
"focus state" attribute value and the "selection state" 
attribute value of the box information. Then, the process unit 
retrieves the "shape" attribute value form the box information 
(S7208), and retrieves the shape information (file name) that 

i 

matches with the "shape" attribute value (S7210), obtaining the 
bit map data corresponding to the file name from the display 
object storage unit 5013 (S7212). Subsequently, the process 
unit retrieves the "character string" attribute value from the 
box information (S7214), and synthesizes the same with the bit 
mfep data obtained in S7212 to generate the display information 
for the box figure (S7216), returning to S7104. 

The process unit additionally stores the button name in 
Jzhe panel information buffer in S7302, and retrieves the button 
information matching with the button name (S7304). The process 
unit retrieves the "focus state" attribute value from the 
button information (S7306) to further retrieve the attribute 
value of the "shape" from tbe button information (S7308). 
Accordingly, the process unit obtains the . shape information 
'(file name) matching with the attribute value of the "shape" 
<S7310), obtaining the bit map data corresponding to the file 
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name from the display object storage unit 5013 (S7312). Then, 
the process unit retrieves the attribute value of the 
"character string" in the button information (S7314), and 
Synthesizes the same with the bit map data retrieved in S7312 
to generate the display information for the button figure 
(«S7316), returning to S7104. 

Subsequently, the process unit successively retrieves 
the object names stored following to the most recently stored 
start position information in the panel information buffer 
(S7402), and virtually places one or more object either 
horizontally or vertically as per position information 
(.coordinate values) in the position data retrieved in S7126 
(S7404). Further, the process unit attaches a name for a set 
of the objects retrieved in S7402 (S7406), and deletes the 
start position information retrieved in S7126 and the objects 
names in S7402 (S7408), and stores the set name in the panel 
information buffer instead (S7410), returning to S7104. 

In this embodiment, the operation related to the 
generation of one interactive screen was explained, and it is 
foell assumed that the operation detailed by the flowcharts S71 
through S74 are repeated as many times as necessary to process 
all the IS structure specification data transmitted to the IS 
generation unit 5014. 

The initial interactive screen as shown in FIG. 63 is 
generated by the IS generation unit 5014 in the above manner. 

Next, in the following, the explanation will be given 
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using the flowchart in FIG. 75 for a case where the user 
transmits a RM signal to the remote controller 6501 (FIG. 65) 
to the program receiver 5002 while referring to the screen 
6401 (FIG. 64). Note that the operation ends when the user 
pushes the "power" button 6502 of the remote controller 6501. 

To begin with, the user pushes the "power" button 6502 
of the remote controller 6501, then the RM signal receipt unit 
5016 waits for the transmission of the RM signal (S7502). 
Given these circumstances, a quiz, "This is the scenery in 
Berlin, the capital of a country in Europe. Now, name the 
country?" is transmitted from the program transmitter 5001 to 
the program receiver 5002. Then, the user pushes the "right" 
button 6504 on the remote controller 6501, so that the "right" 
event signal is transmitted from the remote controller 6501 to 
the program receiver 5002. Upon transmission of the RM signal 
(&7502), the RM signal receipt unit 5016 receives the "right" 
event signal, and transmits the same to the IS generation unit 
5014 (S7504). 

The second receipt unit of the IS generation unit 5014 
receives the "right" event signal, and transmits the same to 
the process unit. Accordingly, the process unit retrieves the 
box information (FIG. 56) from the storage . unit, and further 
retrieves the "America box" whose initial focus state is "ON" 
(S7506) to obtain the action information (FIG. 58) for the 
"America box" from the storage unit (S7508). The process unit 
retrieves the attribute "shift focus" of the "command" 
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corresponding to the "right" event signal, from the action 
information of the "America box" and the attribute of the 
"argument", and "France box" (S7510). The process unit 
retrieves the procedure for the "shift focus" from the basic 
action object storage unit -'5018 • The basic action object 
s.toirage unit 5018 stores the data to switch the focus state 
of the subjected object from "ON" to "OFF" in the "shift 
focus", and switches the focus state of the object specified 
by the "argument" from "OFF" to "ON". Thus, the process unit 
rewrites the box information (FIG. 56) in the storage unit to 
switch the focus state of the "America box" from "ON" to "OFF", 
while switching the focus state of the "France box" from "OFF" 
to "ON", and retrieves the content in the display object 
storage unit 5013 as per shape information (FIG. 60), changing 
the interactive screen as shown in FIG. 66 (S7512). 

Subsequently the process unit returns to S7502, and 
waits for the RM signal from the user. When the user pushes 
1?he "down" button 6505 on the remote controller 6501, the 

» 

"down" event signal is transmitted from the .remote controller 
6501 to the RM signal receipt unit 5016 (S7504). The process 
unit receives the "down" event signal and recognizes that the 
subjected object is the "France box" from the content in the 
storage unit (S7506), and obtains the action information (FIG. 
58) of the "France box" (S7508). Then, the process unit 
retrieves the attribute "shift focus" of the "command" 
corresponding to the "down" event signal and the attribute 
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"German box" of the "argument" from the action information of 
the "France box" (S7510). Then, the process .unit rewrites the 
box information (FIG. 56) in the storage unit, so that it 
retrieves the procedure for the "shift focus" from the basic 
action object storage unit 5018, and switches the focus state 
o£.the "France box" from "ON" to "OFF", while switching the 
•focus sate of the "German box" from "OFF" to "ON" • The process 
unit retrieves the content in the display object storage unit 
5013 as per shape information (FIG. 60), J and changes the 
interactive screen as shown in FIG* 67 (S7512). 

/ The user pushes the "selection" button 6507 on the 

remote controller 6501 to select "Germany" as the answer of the 
quiz, then, the RM signal receipt unit 5016 receives the 
"Selection" event signal (S7504). The process unit recognizes 
that the subjected object is the "Germany box" from the focus 
state (S7506), and obtains the action information of the 
"Germany box" (FIG- 58) (S7508). The process unit retrieves 
the attribute "switch state" of the "command" corresponding to 
the "selection" event signal from the action information of 
the "Germany box" (S7510). Then, the process unit retrieves 
the procedure related to the "switch state" from the basic 
action object storage unit 5018 to rewrite the box information 
( FIG • 56), so that the "selection state" of the "Germany box" 
is switched to "ON". While at the same time, the process unit 
switches the value of the state candidate to "ON", changing the 
interactive screen as shown in FIG. 68 (S7512). 
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Then, the user pushes the "down" button 6505 on the 

! 

remote controller 6501 to shift the focus to the "transmission 

button", and transmits the "down" event signal to the program 

receiver 5002. Subsequently, S7502 through S7512 are repeated, 

* 

and the interactive screen as -shown in FIG. 69 is generated. 

Next, the user pushes' the "determination" button 6508 
on the remote controller 6501 to transmit the answer of the 
quiz "Germany" to the program transmitter 5001. Then, the RM 
signal receipt unit 5016 receives the "determination" event 
signal (S7504), and the process unit of the IS generation unit 
5014 retrieves the same and recognizes that the subjected 
object is the "transmission button" (S7506). 

The process unit retrieves the "transmission button 
action" from the action information (FIG. ! 58 ) (S7508), and 
furjther retrieves the procedure of the attribute of the command 
corresponding to the "determination" event signal, or the 
"transmit attribute value" from the basic action object storage 
unit 5018 (S7510). The process unit retrieves the attribute 
values ("ON" or "OFF " specified by the state candidates) of the 
attribute of the "argument" specified by the event 
"determination" of the "transmission button action" : "America 
.box", "France box", "England box", and "Germany box", and 
transmits the same to the information transmission unit 5017 
(S'7512). 

The information transmission unit 5017 retrieves the 
values of the state candidates and the box names retrieved by 




' ' ' ' ! 

the process unit in S7512, and transmits j the same to the 

I 

.* information receipt unit 5019 of the program jtransmitter 5001. 
• ' j 
A : h In this embodiment, following information! is transmitted: 

: -. y • i j 

I | f "America box -OFF", "France hoic*OFF", "England box • OFF" , and 
■5^ "Germany box*0N". Thus, the* data exhibiting "the answer is 

..v. • ~ • ; 

' ' ** , Germany" is transmitted, and the program ! transmitter 5001 
• specifies the users with . the correct answer, or counts the 
. number of such users. 

: ' i 

As has been stated, the process unit interprets the IS 

■ ■* ! 
id structure specification data from the programs transmitter 5001, 

and generates the interactive screen by synthesizing the same 

i 

with the basic display elements stored in the display object 
storage unit 5013. Thus, the interactive screen can be changed 
by synthesizing the interactive screen data with the actions 
15 stored in the basic action storage unit 5018, j and the resulting 
interactive screen data can be transmitted to the program 
transmitter 5001. 

NINTH EMBODIMENT 
FIG. 76 is view depicting the structure of a program 
20 receiver and a program transmitter in accordance with the ninth 
embodiment. 

A program transmitter 7601 includes a program 
transmission unit 7610 for transmitting the IS structure 
specification data by multiplexing the same. 
25 A program receiver 7602 includes a signal receipt unit 
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5011 , a signal separation unit 5012, a display object storage 
unit 5013, a display unit*5015, a RM signal receipt unit 5016, 
a f basic action object storage unit 5018, an IS structure 
specification data storage unit 7611, a screen synthesis unit 
•7612, a print unit 7613, and an IS generation unit 7614. Like 
components are labeled with like reference numerals with 
respect to the eighth embodiment, and the description of these 
components is not repeated. 

The IS structure specification data storage unit 7611 
stores first IS structure specification data excluding a 
synthesis direction generated by the signal separation unit 
5012. 

The screen synthesis unit 7612 includes a counter that 
attaches a n'th number to a synthesis process unit, synthesis 
storage unit, and a synthesis command. The . synthesis process 
unit receives second IS structure specification data including 
the synthesis direction from the IS generation unit 7614, and 
stores the same in the synthesis storage unit. Also, the 
screen synthesis unit 7612 retrieves the first IS structure 
specification data from the IS structure specification data 
storage unit 7611; the first IS structure specification data 
correspond to the data subject to synthesis included in the 
second IS structure specification data. The screen synthesis 
unit 7612 synthesizes the first and second IS structure 
specification data, and transmits the resulting synthesized 
IS structure specification data to the IS generation unit 7614. 
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When there exist no first IS structure specification data, the 
second IS structure specification data are transmitted to the 
IS generation unit 7614 as the synthesized IS structure 
specification data. 

Assume that the IS structure specification data unit 
7611 stores the first IS structure specification data as was 
explained in the eighth embodiment using FIGS. 54 through 60 
♦ and the screen synthesis unit 7612 receives the first IS 
structure specification data shown in FIGS. 77 through 88. 
Then, the synthesis process unit has the synthesis storage 
unit store the second IS structure specification data: the 
synthesis process unit retrieves the synthesis command per unit 
sandwiched between " <SYNTHESIS_DIRECTION> " and 
" </SYNTHESIS_DIRECTION> " , while resetting the counter to .zero. 
When there exists no synthesis command, the generation of the 
IS structure specification data is terminated. The synthesis 
process unit further checks whether the retrieved synthesis 
command is "add command" or "display command". 

In case of the "add command 11 , the synthesis process unit 
checks whether the subjected information is the object 
information, action information, shape information or display 
candidate information using the information sandwiched by 

" < SYNTHES I S_C0MMAND > ADD " and " < / SYNTHES I S_COMMAND > " . 

f 

The synthesis process unit retrieves the object 
information shown in FIG. 77 from the synthesis storage unit. 
The object data includes an object identifier "quizl", and its 
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attribute value "timeout = 60 sec". The synthesis process 
unit retrieves the object information (panel information in 
FIG. 55) corresponding to the object identifier in the IS 
structure specification data storage unit 7611, and synthesizes 
the same with the attribute 'value "timeout = 60 sec" in the 
"add command" in the second IS structure specification data, 
generating new panel information as shown in FIG.- 81. 
Accordingly, the synthesis process unit transmits the new panel 
information to the IS generation unit 7614, and increments the 

counter by one. 

Then, the synthesis process unit retrieves the synthesis 
command. The retrieved action information comprises, as shown 
in FIG. 78, the action identifier "America box action", event 
"help", execution command "open panel" for the event, and the 
argument of the execution command "help panel". The synthesis 
process unit retrieves the action information ( "America box 
action" in FIG. 58) corresponding to the "American box action" 
specified by the action identifier from the IS structure 
specification data storage unit 7611. Since there exists no 
event "help" for the attribute value of the "America box 
action", the synthesis process unit synthesizes the event, 
execution command and argument, generating the new action 
information "America box action" as shown in FIG. 82. 
Accordingly, the synthesis process unit transmits the same to 
the IS generation unit 7614 and increments the counter by one. 

Assume that the IS structure specification data storage 



unit 7611 stores the " America, ."box action", and there exists the 
definition of the event "Help". Then, the synthesis process 
unit generates the synthesized information by adding 
"<COMMAND>" "open panel" and " < ARGUMENT" > "help panel" to 
11 < COMMAND > "display guidance" of the first IS structure 
specification data, and the resulting IS specification 
structure data are shown in FIG. 84. 

The synthesis process unit retrieves the synthesis 
command from the shape information shown in FIG. 79. The shape 
information includes a shape identifier "button shape", a 
selection of the shape information identified by the .shape 
identifier "selection state = ON", and a figure in compliance 
with the condition "button shape 3". The synthesis process 
unit retrieves the shape information ("button shape" in FIG. 
60) corresponding to the "button shape" specified by the shape 
identifier from the IS structure specification data storage 
unit 7611, and checks whether there exists the condition 
"selection state - ON" in the condition of the button shape. 
Since there is no condition "selection state", the synthesis 
process unit adds the condition to the "button shape", 
generating the synthesized data as shown in FIG. 85. 
Accordingly, the synthesis process unit transmits the 
synthesized data to the IS generation unit 7614, and increments 
the counter by one. 

i 

The synthesis process unit retrieves the display 

.<♦ 

candidate information shown in FIG. 80 from the synthesis 



storage unit. The display candidate information includes a 
display identifier "0NJDFF candidate", arid more than one 
display candidate value "null". The synthesis process unit 
retrieves the display candidate information (FIG. 59) from the 
IS structure specification data storage unit 7611, and since 
there exists "ON_OFF candidate**' corresponding to the display 
identifier, it adds the display candidate value "null", 
generating the synthesized data as shpwn in FIG. 86. 
Accordingly, the synthesis process unit transmits the 
synthesized display candidate information to the IS generation 
unit 7614, and increments the counter by one. 

The synthesis process unit transmits the panel 
identifier and position information to the IS generation unit 
7614 in case of the display command. 

If the add command retrieved by the synthesize process 
unit is neither allowed to be processed nor the display 
command, then, the synthesis process unit transmits an error 
message to the IS generation unit 7614. 

The synthesis process unit retrieves the display 
candidate information shown in FIG. 80 from the synthesis 
storage unit. The display candidate information comprise a 
display identifier "0NJ3FF candidate" and more than one display 
candidate value "null". The synthesis process unit retrieves 
the display candidate information (FIG.. 59) from the IS 
structure specification data storage unit 7611, and since there 
exists "ON OFF candidate" corresponding to the display 
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identifier, the synthesis .process unit adds the display 
candidate value "null" to ' the retrieved display candidate 
information, generating new display candidate information as 
shown in FIG. 86. Accordingly, the synthesis process unit 
transmits the same to the IS generation unit, and increments 
the counter by one. 

The synthesis process unit, in case of the display 
command, transmits the panel identifier and position 
information to the IS generation unit 7614. When the add 
command retrieved by the synthesis process unit is neither 
allowed to be processed nor the display command, then the 
synthesis process unit "transmits an error message to the IS 
generation unit 7614. 

The print unit 7613 receives a print direction from the 
IS ^generation unit 7614, and prints out the interactive screen 
currently displayed on the display unit 5015. 

The IS generation unit 7614 includes a third receipt 
unit for receiving the data from the screen synthesis unit 
7612 in addition to the structure of the IS generation unit 
5014 of the eighth embodiment. 

The process unit is designed to function as follows in 
addition to the function of the process unit of the eighth 
embodiment. The first receipt unit checks whether the IS 
structure specification data from the signal separation unit 
5012 includes the synthesis information or not. When the 
synthesize information is excluded, the process unit has both 
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the IS structure specification data storage unit 7611 and 
storage unit store the IS structure specification data as the 
first IS structure specification data; otherwise, it transmits 
the same to the screen synthesis unit 7612 as the second IS 
structure specification data? Upon receipt of the error 
message from the third receipt unit, the process unit directs 
the display control unit to display "error message" indicating 
inadequate synthesis command . 

Also, the' process unit receives the event signal from 
the RM signal receipt unit 5016 by way of the second receipt 
unit, then it transmits a print direction to the print unit 
7613 when the basic action object storage unit 5018 does not 
store the basic action corresponding to the command name. 

The third receipt unit receives the synthesized IS 
structure specification data from the screen synthesis unit 
7612, and rewrites the IS structure specification data in the 
storage unit. The third receipt unit also notifies the error 
to the process unit upon receipt of the error message from the 
screen synthesis unit 7612. 

Next, an example of the first IS structure specification 
data from the signal separation unit 5012 will be explained 
using FIGS. 87 through 92. 

FIG. 87 defines the attribute of the class shown in FIG. 
54 in the eighth embodiment. Here, "<! ELEMENT TEXT ( CHARACTER^ 
STRING )>" defines that the object belonging to the "text" class 
has the attribute "character string". ! 



FIG. 88 defines the attribute of the objects belonging 
-to the panel class composing the interactive screen, and the 
objects of the panel. Here, "user panel 1" includes "address 
1"V "telephone no. 1", "hot spring 1", and "print button", and 
the "shape attribute" is "panel shape". 

FIG. 89 defines the objects belonging to the text class. 
For example, the object of "address 1" displays the character 
string "Shirahama-cho, Wakayama Pref.", and the object of the 
"telephone no. I" displays the character string "059352-1- 
201"; the object of the "hot spring 1" displays the character 
string "Shirahama Hot Spring". 

FIG. 90 defines the objects belonging to the button 
class. The "print button" object displays the character string 
"print", its shape is defined as the "button shape", the action 
is defined as the "print button action", and the focus state 
at the initialization is "ON" . 

FIG. 91 shows the information that defines the action 
of the objects belonging to the button class. "Print button 
action" defines that the "print" command is executed when the 
"determination" event is transmitted. In executing the 
"print", the arguments "address 1", "telephone no. 1", and "hot 
spring 1" are supplied. If the subjected object is the "print 
button", then the "address 1", "telephone no. 1", "hot spring 
1" are printed out when the "determination" event is 
transmitted . 

FIG. 92 defines the shape information: the information 
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sandwiched by "<SHAPE>" and "</SHAPE>" are the definition of 
the shape information, and the shape information varies 
depending on the definition of " <CONDITION> " . More precisely, 
the "panel shape" always appears as the "panel shape 1", and 
the "button shape" appears as the "button shape 1" given the 
condition "focus state = OFF", while appearing as the "button 
shape 2" given the condition "focus state = ON" . 

As has been stated, by receiving the first IS structure 
specification data, the display unit 5015 displays the 
interactive screen as shown in FIG. 93 on the window 6402. 

Further, the signal receipt unit 5011 receives the 
multiplex signal of the first IS structure specification data 
(FIGS. 95 through 99) including the second IS structure 
specification data (FIG. 94). 

FIG. 94 shows the synthesis direction' information . The 
•synthesis direction includes a plurality of synthesis commands, 
and the synthesis command includes an add command and a display 
command . 

The information sandwiched by " <SYNTHESIS JDIRECTION> " 
and "</SYNTHESIS_DIRECTION>" are the synthesis direction. 
"<SYNTHESIS_COMMAND>DISPLAY" means to synthesize the objects 
sandwiched by " <SYNTHESIS__COMMAND> DISPLAY" and " </SYNTHESIS_ 
COMMAND > " , and places the synthesized objects. Since two 
objects are sandwiched between "<v>" and "</v>", the two panel 
objects "user panel 1" and "user panel 2" are placed 
vertically.. 
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M <SYNTHESIS_COMMAND>ADD M includes the- information to be 
synthesized and the parameter indicating the synthesizing 
information, and it is the command to add the information 
sandwiched between the " <synthesis_COMMAND>add" and 
5. " <SYNTHESIS COMMAND > " to the first IS structure specification 
data. This means to add the parameters 

" < EVENT > DOWN < COMMAND > SHI FT_FOCUS > ARGUMENT > PRINT_BUTT0N 2" to 
the "print button action" specified by 
"<ACTION>PRINT_BUTTON_ACTION" in the action information (FIG. 
10 91) of the first IS structure specification data. By 
" <SYNTHESIS__COMMAND> " , the action information shown in FIG. 
100 is generated. 

FIG. 95 defines the objects belonging to the panel 

class. 

15 FIG. 96 defines the objects belonging to the text class. 

FIG. 97 defines the objects belonging to the button 

class. 

FIG. 98 is the data defining the action of the objects 
belonging to the button class. 

i 

20 FIG. 99 defines the shape information. 

The synthesized IS structure specification data are 

generated by the screen synthesis unit 7612 and the display 

unit 5015 displays the interactive screen as shown in FIG. 101 

on the window 6402. j 
25 When the user pushes the "down" button 6505 on the 

remote controller 6501, the interactive screen as shown in FIG. 
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102 appears on the window 6402* 

Next, the operation related to this embodiment will be 
explained while referring to the flowcharts in FIGS. 103 
through 105. 

FIG. 103 is the main flow of the program receiver 7602. 

The signal receipt unit 5011 receives the signal 
(S10302), and the signal separation unit 5012 checks whether 
the received signal is a multiplex signal or not (S10304). 
In case of the multiplex signal, the signal separation unit 
5012 separates the signal into the information supply program 
signal and the IS structure specification data signal (S10306); 
otherwise, it transmits the signal to the display unit 5015 
(S10308). Further, the signal separation unit 5012 checks 
whether the signal is the information supply program signal 
or not (S10310), and in case of the information supply program 
signal, it transmits the same to the display 5015 (S10308). 

When the received signal is not the information supply 
prq^ram in S10310, the signal separation unit 5012 checks 
whether there exist the IS structure specification data 
(S10312). If there is no IS structure specification data, it 
proceeds to S10308 to generate the screen for the information 
supply program at the display unit 5015 (S10309); otherwise, 
it transmits the signal to the IS generation unit 7614, which 
checks whether the IS structure specification data include the 
synthesis direction or not (S10314). If the synthesis 
direction is included, the IS generation unit 7614 transmits 
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the IS structure specification data including the synthesis 
direction to the screen synthesis unit 7612 as the second IS 
structure specification data (S103016). Otherwise, it has the 
IS structure specification data storage unit 7611 and the 
5 storage unit in the IS generation unit 7614 store the IS 
» structure specification data excluding the synthesis direction 

as f the first IS structure specification data (S10318). The 
screen synthesis unit 7612 synthesizes the second IS structure 
specification data and the first IS structure specification 
10 data in the IS structure specification data storage unit 7611 
as per synthesis direction, generating new IS structure 
specification data (S10320). 
f / The screen synthesis unit 7612 transmits the synthesized 

IS structure specification data to the IS generation unit 7614 
5- 15 (S10322). 

The IS generation unit 7614 interprets the IS structure 
• specification data, and synthesizes the same with the basic 

display elements in the display objects storage unit 5013 to 
generate the interactive screen (S10324). 
20 The display unit 5015 displays the information supply 

program generated in S10309, and displays the interactive 
screen generated by the IS generation unit 7614 while the 
interactive screen is being generated in S10324 (S10326). 

Further, the process unit checks whether the signal 
25 receipt should be stopped or not (S10328)., If it should be 
stopped, the process unit ends the operation; otherwise, it 
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returns to S10302. 

Next, S10320 will be explained more, in detail while 
referring to the flowchart in FIG. 104. 

The screen synthesis unit 7612 retrieves the synthesis 
direction in the second IS structure specification data from 
the interactive screen generation unit 7614, and resets the 
counter to zero (n=0) (S10402), and checks whether there exists 
the n ! th command or not (S10404). If there exists the n'th 
command, the screen synthesis unit 7612 retrieves the n'th 
synthesis command in the synthesis direction (S10406); 
otherwise, it ends the operation* 

Having retrieved the synthesis command, the screen 
synthesis unit 7612 checks whether the synthesis command is the 
addition command or not (S10408). In case of the addition 
command, the screen synthesis, unit 7612 further checks whether 
the object information is subject to synthesis or not (S10410). 
If the synthesis command is not the addition command, the 
screen synthesis unit 7612 proceeds to S10412; otherwise, it 
synthesizes the object information while adding one to n 
(S10414), and returns to S10404, If the object information is 
not subject to synthesis, the screen synthesis unit 7612 checks 
whether the action information is subject to synthesis or not 
(S10416). If so, the screen synthesis unit 7612 synthesizes 
the action information while adding one to n (S10418) and 
returns to S10404. If the action information is not subject 
to synthesis, the screen synthesis unit 7612 checks whether 



the shape information is subject to synthesis or not (S10420). 
If so, the screen synthesis unit 7612 synthesizes the shape 
information while adding one to n (S10422), and returns to 
S10404; otherwise, it further checks whether the display 
candidate information is subject to synthesis or not (S10424). 
If so, the screen synthesis unit 7612 synthesizes the display 
candidate information while adding one to n (S10426), and 
returns to S10424; otherwise, it checks whether the synthesis 
command is the display command or not (S10412). In case of the 
display command, the screen synthesis unit 7612 notifies the 
panel identifier and position information in the synthesis 
command to the IS generation ,unit 7614 while adding one to n 
(S10428), and returns to S104Q4. Otherwise, it displays "error 
message" indicating an inadequate synthesis command on the 
display unit 5015 and ends the operation. 

Next, how the user manipulates the interactive screen 
in S10326 will be explained while referring to the flowchart 
in FIG. 105. 

Two hot springs "Shirahama Hot Spring" and "Arima Hot 
Spring" are introduced in the program as shown in fig. 101. 
Now, the object (focused "ON" object) accepting the signal 
from the remote controller 6501 is the "print button" object 
placed on the panel introducing "Shirahama Hot Spring", and the 
user pushes the down button 6505 on the remote controller 6501 
if he wants the information be printed. Then, the "down" event 
signal is transmitted to the program receiver 7602 from the 

i 
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remote controller 6501. 

The RM signal receipt unit 5016 waits for the RM signal 
from the remote controller 6501 (S10502), and receives the 
"down" event signal, transmitting the same to the IS generation 
unit 7614 (S10504). 

The IS generation unit 7614 receives the "down" event 
signal, and accordingly retrieves the object with the "focus 
state = ON" from the storage unit, and recognizes that the 
"print button" is the object subject to manipulation, and thus 
retrieving the button information "<BUTT0N CHARACTER_STRING = 
PRINT • • • ACTION = PRINT_BUTT0N__ACTI0N • • • > PRINTJ3UTT0N" defined 
as FIG. 90 (S10506). Then, the IS generation unit 7614 obtains 
the action name of the retrieved button ( action information ) , 
"print button action (FIG, 91)(S10508). The IS generation unit 
7614 retrieves "<EVENT>DOWN < COMMAND > SHI FT_F0CUS 
<ARGUMENT>PRINT_BUTT0N2" from the "down" event signal and the 
information defined by "print button action" (FIG. 100) 
(S10510). 

The IS generation unit 7614 interprets the action 
composed of the commands and arguments, and retrieves the basic 
action from the basic action object storage unit 5018 (S10512, 
S10514), effecting the basic action. Thus, the focus is 
shifted to the "print button 2", and the interactive screen as 
shown in FIG. 102 appears (S10516), and the IS generation unit 
7614 returns to S10502. 

The user pushes the "determination" button 6508 on the 
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remote controller 6501 to print out the information as to 
"Arima Hot Spring". Accordingly, the "determination" event 
signal is transmitted to the program receiver 7602 from the 
remote controller 6501 (S10502). The RM signal receipt unit 
5 5016 receives the "determination" event signal, and transmits 
the same to the IS generation unit 7614 (S10504). Upon receipt 
of the "determination" event signal, the IS generation unit 
7614 extracts the object with the "focus state = ON" from the 
storage unit, and recognizes that the "print button 2" is the 

10 object subject to manipulation, and thus retrieving the button 
information " <BUTT0N CHAR ACT ER_S TR I NG = PRINT- • • ACTION = PRINT_ 
BUTTON2 • • • >PRINT_BUTT0N 2" defined as FIG. 97 (S10506). 

The IS generation unit 7614 retrieves the action 
information "print button action 2" from the storage unit using 

15 the action name "print button action 2" in the retrieved button 
information (S10508). 

The IS generation unit 7614 retrieves 
" < E VENT > DET ERMIN AT I ON <C0MM AND >PRI NT < ARGUMENT > ADDRESS 
2 < ARGUMENT > TELEPHONE NO. 2<ARGUMENT>H0T_SPRING 2" (FIG. 98) 

20 from the "determination" event signal (S10510). 

The IS generation unit 7614 retrieves the basic action 
.corresponding to the command name "print" from the basic action 
object storage unit 5018 (S10512). If there exists no such 
' basic action (S10514), the IS generation unit 7614 checks 

25 whether the action is the "print" or not (S10518). If the 
action is not the "print", the IS generation unit 7614 returns 
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to S10502; however, since the action is the "print" herein, it 
proceeds to S10520. 

The IS generation unit 7614 delivers "Arima-cho, Hyogo 
Piref.", "07234-1-101" , and "Arima Hot Spring", which are 
specified by the "argument" of "address 2", "telephone no* 2", 
and "hot spring 2", respectively, to the print unit 7613 
(S10520). 

The print unit 7613 prints out the received information 
(S10522), and returns to S10502. This action is repeated until 
the power 6502 of the remote controller 6501 is turned off. 

As has been explained, a plurality of IS structure 
specification data are synthesized as per synthesis command 
from the program transmitter 7601, and by including the action 
specification information in the IS structure specification 
data, the user can manipulate the program receiver 7602 with 
the interactive screen. 

In this embodiment, the display object, storage unit 5013 
stores three kinds of basic figures: panel, box and button. 
However, the display object storage unit may be a floppy disk 
or CD-ROM, so that the display may be changed for individual 
users. 

Although the print unit 7613 is furnished, it may be 
replaced with a magnetic memory. 

The generation of the interactive screen for the TV was 
explained in this embodiment. However, the interactive screen 
may be generated on the LC display with a radio using the text 
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broadcast system by FM broadcast. 



TENTH EMBODIMENT 

FIG. 106 is a view depicting the structure of a 
graphical interactive screen (GIS) display unit and an 
appliance that specifies the display of the graphical 
interactive screen unit in accordance with the tenth embodiment 
of the present invention. 

An appliance, or a VTR '10620, comprises a GIS structure 
data transmission unit 10621, a user's GIS structure data 
storage unit 10622, a manipulation direction data transmission 
unit 10624, a control unit 10625, and a manipulation unit 
10627. 

A GIS display unit 10610 comprises a GIS structure data 
receipt unit 10611, a display's GIS structure data storage unit 
10612, a graphical display element storage unit 10613, a 
manipulation direction data receipt unit 10614, a GIS 
generation unit 10615, and a graphical display unit 10616. 

The GIS structure data transmission unit 10621 of the 
VTR 10620 and the GIS structure data receipt unit 10611 of the 
GIS display unit 10610 are connected via a transmission path 
10623. Likewise, the manipulation direction data transmission 
unit 10624 and the manipulation direction data receipt unit 
10614 are connected via a transmission path 10628. The 
transmission paths 10623 and 10628 are, for example, cables, 
ultra red rays, and connecter sockets. 
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The user ' s GIS structure data storage unit 10622 stores 
in advance the GIS structure data related to the setting of 
tuner in the VTR 10620, an example of which is shown in FIGS, 

108 through 117, Note that one drawing is developed in FIGS . 

109 through 112. The GIS structure data are written using the 
grammar of SGML ( Standard Generalized Markup Language ) arranged 
by the object oriented technology. The SGML is an 
international standard language for electronic books or 
documents stipulated by the ISO (International Standardization 
Organization). More detailed explanation of the SGML is given 
in "Introduction to SGML" , Aski Shuppan. 

Each display element is displayed as an object, and they 
are divided into a set of hierarchy groups. Entity class is 
the high-order class, and the first hierarchy includes the 
"panel", "button", and "box" classes while the second hierarchy 
includes the "set box" class. 

FIG. 108 defines each class. In the "panel" class 

10801, a parent class, the action, shape and focus state are 

defined. The attribute value of the parent class is the entity 

class; for there exists the entity class in the high-order 

hierarchy as shown in FIG. 107. The attribute value of the 

action is the "panel action". Thus, when, the panel is the 

subject to manipulation, the "panel action" is executed when 
* 

the "event" signal is transmitted. The action attribute is 
defined in FIGS. 109 through 112. The attribute value of the 
shape is "panel shape". The shape attribute is defined in 
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FIG. 113- The attribute value of the focus state is "ON", 
which means it is the object subject to manipulation when the 

"event" signal is transmitted. 

f * 

In the "button" class 10802, the attribute of the parent 

f • 

5 clVss> action, shape, character string, focus state and set 
box is defined. The initial attribute value of the "character 
string" shown in FIG. 114 defines each "button". The attribute 
value of the "set box" defines the box forming a pair with the 
7* "button", and the. value in the "button" class 10802 is empty. 
10 The "box" 10803 defines the attribute of the parent 

class and shape. 

The "set box" class 10804 defines the attribute of the 
parent class, display candidate, action and the set value. 
The attribute value of the "display candidate" is the "channel 
15 candidate". The candidate value to be set as the "channel 
candidate" is shown in FIG. 115, and the attribute value of the 
"set value" is shown in FIG. 116. 

The value "1" is defined as the "set value" for the "set 
1 box", while the value "2" is defined as the "set value" for 
20 the "set 2 box". Similarly, the value "BS11" is defined for 
the "set 15 box". 

FIG. 117 shows the definitions of the object belonging 
.to the panel class forming the graphical interactive screen. 

" < PANEL ACTION = PANE L__ACT I ON SHAPE = PANEL_jSHAPE>TUNER_ 
.25 SETJPANEL" defines the "tuner set panel" object shown in FIG. 

113 as follows: it belongs to the panel class, and the 
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attribute value of the "action" is the panel action shown in 
FIG. 109, and the attribute value of the "shape" is the panel 
shape shown in FIG. 113. Note that the action and shape have 
been defined as the attribute of the panel class as shown in 
FIG. 108, it may be " <PANEL>TUNER__SETJPANEL" instead. Here, 
the objects in the panel Sandwiched by " < PANEL • • • > " and 
"</PANEL>" are defined. 

Also, "<v>", "</v>", "<h>", "</h>" define the position 
of the objects. The objects sandwiched by "<v>" and "</v>" 
are placed vertically, while the objects sandwiched by " <h>" 
and "</h>" are placed horizontally. That is to say, the "set 
1 button" defined in FIG. 114 and the "set 1 box" defined in 
FIG. 116 are placed horizontally as a pair. Similarly, a pair 
of the "set 15 button" and "set 15 box", and a pair of the 

"set 2 button" and the "set 2 box" are placed horizontally. 
These pairs are placed vertically and in the end, the "end 
button" is placed. 

The manipulation unit i0627 of the VTR 10620 includes 
a> "tuner set" button, a "down" button, a "right" button, and 
•a "determination" button, etc. 

When the user pushes the "tuner set" button, the 
manipulation unit 10627 transmits a "tuner set" signal to the 
control unit 10625. Similarly, when the user pushes the "down" 
button and "determination" button, the manipulation unit 10627 
transmits a "down" signal and a "determination signal", 
respectively. 
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The control unit 10625 receives the "tuner set" signal 
from the manipulation unit 10627, and initiates the control of 
the tuner setting, while sending a direction to the GIS 
structure data transmission unit 10621 to transmit the GIS 
structure data to the GIS display unit 10610, transmitting a 
"tuner set" event signal to the manipulation direction data 
transmission unit 10624, Upon receipt of the "down" signal 
from the manipulation unit 10627, the control unit 10625 
initiates the tuner set action related to the pushing of the 
down button (changes "set n button" to "set (n+1) button", 
where n » 1 through 14), while transmitting the "down" event 
signal to the manipulation direction data transmission unit 
lp624. Upon receipt of the "right" signal from the 

manipulation unit 10627, the control unit 10625 changes to the 
tuner set action related to the pushing of the right button 
(changes "channel n" to "channel (n+1)", where n = 1 through 
11, and channel 12 is changed to channel BS5 , BS5 is changed 
to BS7, and BS7 is to BS11), while setting the "right" event 
signal to the manipulation direction data transmission unit 
10624, 

Upon receipt of the "determination" signal form the 
manipulation unit 10627, the control unit 10625 ends the tuner 
set action, and transmits the "determination" event signal to 
the manipulation direction data transmission unit 10624. 

The GIS structure data transmission unit 10621 receives 
the transmission direction from the control unit 10625, and 
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retrieves the GIS structure d,ata (FIGS. 108 through 117) from 

the user's GIS structure data storage unit 10622 to transmit 

the same to the GIS structure data receipt unit 10611 of the 

GIS display unit 10610 via the transmission path 10623. 

The manipulation direction data transmission unit 10624 

receives the "tuner set" event signal from the control unit 

10625, and in response transmits the same to the manipulation 

direction data receipt unit 10614 of the GIS display unit 

101610 via the .transmission path 10628. Upon receipt of the 

"down" event signal from the control unit 10625, the 

manipulation direction data transmission unit 10624 transmits 
* 

the same to the manipulation direction data receipt unit 10614 
via the transmission path 10268. Likewise, upon receipt of the 
"right" event signal and the "determination" signal from the 
control unit 10625, the manipulation direction data 
transmission unit 10624 transmits the same to the manipulation 
direction data receipt unit 10614 via the transmission path 
10628. 

The GIS structure data receipt unit 10611 of the GIS 
display unit 10610 receives the GIS structure data from the 
GIS structure data transmission unit 10621, and in response has 
the display's GIS structure data storage unit 10612 store the 
same. 

The display 1 s GIS structure data storage unit 10612 
comprises such as RAM (Random Access Memory), and stores the 
GIS structure data (FIGS. 108 through 117). 
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The manipulation direction data receipt unit 10614 
receives the manipulation direction data - "tuner set" event, 
"down" event, "right" event, "determination", event signals - 
;£rom the manipulation direction data transmission unit 10624, 
and in response transmits the same to the GIS generation unit 
10615. 

The graphical display element storage unit 10613 stores 
the file name of each shape of the graphical display elements 
(panel, button, and box) and their respective bit map data. 

The GIS generation unit 10615 includes a buffer to 

temporarily store the event signal from the manipulation 

direction data receipt unit 10614, The GIS generation unit 

10615 retrieves the "tuner set" event signal from -the buffer 

and.' finds out the "focus-on" object in -the display's GIS 

structure data storage unit 10612 using the definition in FIG. 

108. Here, the "panel" class is the "focus-on" object, and 

thus the GIS generation unit 10615 extracts the attribute of 

the "panel", finds out the action of the panel, or the "panel 

action", using the definitions shown in FIGS,. 109 through 112, 

and further finding -the "panel display" (FIG. 109), the 

"command" for the "tuner set" event. To execute the "command", 

the GIS generation unit 10615 retrieves the : "tuner set panel" 

whose action's (FIG. 117) attribute value is the "panel 

action". The GIS generation unit 10615 finds the attribute 
ft 

value of the "shape" of the "tuner set panel", or the "panel 
shape A", using the definition of the "shape" attribute in 

136 



FIG. 113, and retrieves the bit map data (11801 in FIG. 118) 
corresponding to the file name "panel shape A" from the 
graphical display element storage unit 10613. 

Next, the GIS generation unit 10165 extracts the "set 
Ij button" "set 1 box", "set 2 button", "set 2 box", "end 
button" to be placed on the panel: it retrieves the definitions 
of the "set 1 button" through "set 15 button" and "end button" 
in FIG. 114 and the definitions for the "set 1 box" through 
"set 15 box" in FIG. 116 from the display's GIS structure data 
storage unit 10162. Thus, the shape of the "set 1 button" is 
the "button shape B" given "focus state « ON", and the shape 
of the "set button 2" through the "end button" are the "button 
shape A" (FIG. 113); for they are not defined by the attribute 
value of the "focus state", and its default is "focus state « 
OFF (the definition of the "button" in FIG. 108 is inherited). 

Likewise, the shape of the "set 1 box" through the "set 15 
box" is the "box shape 1" (FIG. 113). 

The "set 1 button" object is generated by synthesizing 
the bit map data (11803 in FIG. 118) for the file name "button 
shape B" in the. graphical display element storage unit 10613, 
and the set value "1" of the attribute value of the character 

string shown in FIG. 114. The "set 1 box" is generated by 

s 

synthesizing the bit map data (11804 in FIG. 118) for the 
"button shape A" and the set value "2". The "set 2 box" 
through "end button" are generated in the same manner, and in 
the end, the objects thus generated are placed as specified in 
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FIG* 117, and displayed on the graphical display unit 10616 as 
shown in FIG- 119. 

The GIS generation unit 10615 retrieves the "down" event 
signal form the buffer, and finds out the object with the 
"fodus state = ON" in the display's GIS structure data storage 
unit 10612. Assume that the graphical interactive screen as 
shown in FIG. 119 is being displayed on the graphical display 
unit 10616, then the subjected object is the "set 1 button" 
(FIG. 114). As per the definition of the "set 1 button action" 
in FIG. 119, it is found that: the command for the "down" event 
is "focus OFF", and the message is "focus ON", and the 
transmission destination is the "set 2 button". Accordingly, 
the GIS generation unit 10615 rewrites the set value of the 
"set 2 button" to "focus state = OFF", generating the internal 
event of the "focus state = ON" for the "set 2 button". Then, 
the GIS generation unit 10615 retrieves the internal event 
queue to additionally store the set value of the "set 2 button" 
(FIG. 114) to the "focus state = ON". Note that since -there 
is no action definition for the "focus-on" object, which is the 
action attribute defined by the "set 2 button", the "set 2 
button" is set as the "focus state = ON" using the definition 
of the "button" class to which the "set 2 button" belongs. 
Accordingly, the graphical display unit 10616 displays the 
graphical interactive screen shown as FIG. 120. 

When the GIS generation unit 10615 retrieves the "right" 
event signal from the buffer, it finds out the object with the 
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"focus state » ON" in the display's GIS structure data. 

When the graphical display unit 10616 displays the 
graphical interactive screen as shown in FIG. 120, the "set 2 
button" is "focus state = ON", hence becoming the subject of 

the manipulation. However, tfcfe "right" event is not defined 
by the definition of the "set 2 button action" (FIG. 109), thus 
the definition of the button action of the "button" class (FIG. 
109) to which the "set 2 button" belongs is applied. The 
message for the. "right" event is the "switch candidate in plus 
direction" and the transmission destination is "$set box". 
Here, the 11 $ M mark in the transmission destination means the 
description following the mark, "set box", is the attribute 
name, and the subjected object is replaced with the "set box" 
represented by this attribute name. It is found that the 

"attribute value of the "set box" is the "set 2 box" from the 
definition of the "set 2 button" in FIG. 114, and thus the 
internal event "switch candidate in plus direction" represented 
by the "message" occurs for the "set 2 box". Having retrieved 
the definition of the set box in FIG. 116, it is found that the 

command for the "switch candidate in plus direction" event is 

i 

the "switch candidate", and the "argument" is "+". Because 
the definition of the "set box" (10804 in FIG. 10804) is 
.defined as the "display candidate « channel candidate", the 
candidate value for the channel candidate in FIG. 115 is 
applied. Having switched the candidate with the "argument +", 
the candidate value of the current "set 2 box" exhibits "2" is 
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switched to the next • candidate value at the plus side 

exhibiting "3". Accordingly, the set value (FIG. 114) of the 

"set 2 box" in the display's GIS storage unit 10612 is 

rewritten from "2" to "3", and the graphical display unit '10616 

displays the graphical interactive screen as shown in FIG. 121. 
» 

When the GIS generation unit 10615 retrieves the 
"determination" event from the buffer, it retrieves the "end 
button action" (FIG* 112) from the display's GIS structure data 
storage unit 10512 when the "end" button of the graphical 
interactive screen on the graphical display unit 10616 exhibits 
the "focus state = ON" as shown in FIG. 122* Since the 
"message" for the "determination" event is delete, the 
transmission destination is the "tuner set panel", and the GIS 
generation unit 10615 generates the internal event "delete", 
and retrieves the panel action as shown in FIG. 109. Further, 
the GIS generation unit 10615 retrieves the "delete" event from 
the internal event queue, and executes the corresponding 
Command, "delete panel". Accordingly, the display of the 
graphical interactive screen on the graphical display unit 
10616 ends. 

The graphical display unit 10616 is, for example, an LC 
panel or CRT, and displays the graphical interactive screen 
under the control of the GIS generation' unit 10615, and 
examples of the interactive screens are shovfn in FIG. 119 
through 122. 

The operation related to this embodiment will be 
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explained while referring to the flowcharts in FIGS. 123 and 
124. 

The GIS generation unit 10615 retrieves the event queue 

from the buffer sent from the manipulation unit 10625 (S12302), 

and' in case of the event signal (S12304), it finds the " focus - 
t ' 

pri" object by retrieving the GIS structure data from the 
display's GIS structure data storage unit 10612 (S12306). In 
finding the " focus-on M object (S12308), the GIS generation unit 
10615 finds the 'action definition from the object's action 
attribute (S12310); otherwise (S12308), it returns to S12302. 
Further, the GIS generation unit 10615 finds out the definition 
for the corresponding event process in the action definition 
(S12312). In finding the definition (S12314), it carries out 
the event process (S12318) and returns to S12402; otherwise 
(S12314), it retrieves the attribute value of the class to 
which the object belongs (S12316), and returns to S12310. If 
there is no definition for the class's attribute value, the GIS 
generation unit 10615 finds out the parent class reflectively, 
and returns to S12310. Note that the class attribute in the 
general object oriented technology is inherited during this 
process. 

The GIS generation unit 10615 retrieves the internal 
event queue (S10402), and if * there exists the internal event 
'(S12404), it retrieves the object of the transmission 
destination of the event (S12406), When; there exists no 
internal event, it returns to S12302 (S12404). When the object 
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of the transmission destination is specified (S12408), it -finds 
the action definition of the object (S12410), and further finds 
the process definition to the event in the action definition 
(S12412). If there exists no process definition, it returns 
to S12402; otherwise, it carries out the event (S12416), and 
returns to S12402 (S12414). 

When the object of the transmission destination is not 
specified in S12408, then the GIS generation unit 10615 finds 
out the "focus-on" object (S12418). If there exists no "focus- 
on 11 object, it returns to S12402 (S12420); otherwise, it finds 
the action definition from the action attribute in the object 
(S12422), further finding the process definition for the event 
in the action definition (S12424). When finding the process 
definition (S12426), the GIS generation unit 10615 carries out 
the event process (S12430), and returns to S12402; otherwise, 
it retrieves the attribute value of the class to which the 
object belongs (S12428), and returns to S12422. If there 
exists no definition for the class's attribute vale, then it 
,?finds the parent class reflectively to return to S12422. 

Although the manipulation unit 10627 is incorporated in 
the VTR 10502, a RM manipulation receipt unit may be 
additionally furnished with the VTR 10620, and the manipulation 
unit 10627 may be a remote controller. In this case, the 
transmission form the remote controller is received by the RM 
manipulation receipt unit, which transmits the same to the 
control unit 10625. 
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ELEVENTH EMBODIMENT 

FIG, 125 is a view depicting a graphical interactive 
screen (GIS) display unit and an appliance that directs the 
interactive screen display in accordance with the eleventh 
embodiment. Hereinafter, like components are labeled with like 
reference numerals with respect to the tenth embodiment, and 
the description of these components is not repeated. 

The selection manipulation for speech, video menu, and 
speech menu with a VTR 12501 will be explained as an example. 

A user's GIS structure data storage ,unit 12502 stores 
in advance the class definition shown in FIG. 108 and shape 
definition in FIG. 109, which are explained in the tenth 
embodiment, and the IS structure data shown in FIG. 126 through 
129 . Note that one drawing is developed in FIGS. 126 through 
,L29 . 

FIG. 126 defines that the attribute value of the action 
o£ the "adjustment panel" , which is the object belonging to the 

A 

panel class forming the graphical interactive screen, is the 
adjustment panel action (FIG. 128). Above ;the adjust panel, 
there exist "speech switch button" and "speech switch box" 
horizontally followed by "video menu button" and "video menu 
box", and the "sound menu button" and "sound Jmenu box", and the 

i 

"end button" at the end. 

FIG. 127 defines the objects belonging to the button and 
box classes. 
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* FIG. 128 shows the action definition of the adjustment 
panel or the like. 

FIG. 129 defines the candidate values of the "speech 
switch box 11 in the "set box class", "video menu box", "switch 
display" attribute in the "speech menu box". 

The control unit 12503 receives the "video speech 
.selection" signal from the control unit 10627, and in response 
initiates the control of the video speech selection action, 
while sending a direction to the GIS structure data 
transmission unit 10621 to transmit the GIS structure data to 
the GIS display unit 12504. 

The control unit 12503 examines the current content of 
the appliance action control, and transmits the GIS structure 
data shown in FIG. 130 as subsidiary data to the manipulation 
direction data transmission unit 10624 together with a 
"parameter adjustment event" signal. 

The subsidiary data in FIG. 130 exhibits that the 

program is a bilingual broadcast. Thus, the subsidiary data 

i 

become "display candidate = stereo" when the program is 
transmitted in stereo broadcast, while "display candidate = 
monaural" when the program is transmitted in monaural 
broadcast . ! 

9 This corresponds to the "display candidate = f r " in the 

definition of the "speech switch box" in the GIS structure data 
in FIG. 127, and since the attribute value is empty here, the 
attribute value of the display candidate is specified by the 
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subsidiary data. 

The manipulation direction data transmission unit 10624 

receives the "parameter adjustment event:" signal and subsidiary 
7 

/.data from the control unit 12503, and in response transmits the 
same to the manipulation direction data receipt unit 10614 of 
the G1S display unit 12504 via the transmission path 10628. 

The GIS generation unit 12505 receives the "parameter 
adjustment event" signal and subsidiary data from the 
manipulation direction data receipt unit 10614, and in response 
has the buffer store the same. When it retrieves the 

"parameter adjustment event" signal from the buffer, it 

i 

examines the GIS structure data (FIGS. 108, 109, 126 through 
129) in the GIS structure data storage unit 10612, and finds 
the "focus state = ON" object from the class definition (FIG. 
108), carrying out the "display panel" command corresponding 
to the "parameter adjustment event" defined in FIG. 128. In 
executing the command, it retrieves the "bilingual" from the 
buffer, which is displayed as the "switch box" as the display 
candidate, and synthesizes the same with the bit map data of 
the box shape in the graphical display element storage unit 

10613, generating the speech switch box. ■ Accordingly, the 

i 

graphical display unit 10616 displays the interactive screen 
as shown in FIG. 131 as the "adjustment panel". 

The operation related to this embodiment is identical 
.with that of the tenth embodiment except that the attribute 
value of the display candidate is transmitted as the subsidiary 
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data without being stored in the display's GIS structure data 
storage unit 10612, and stored in the buffer in the GIS 
generation unit 12505. Thus, the further explanation is 
omitted. 

4 

7 , TWELFTH EMBODIMENT 

FIG. 132 is a view depicting a graphical interactive 
screen (GIS) display unit in accordance with the twelfth 
embodiment of the present invention. Hereinafter, like 
components are labeled with like reference numerals with 
respect to the tenth embodiment, and the description of these 
components is not repeated. 

A GIS display unit 13201 includes a TV manipulation unit 
13202, a TV control unit 13203, and a TV GIS structure data 
storage unit 13204 in addition to the structure of the GIS 
display unit 10610 in the tenth embodiment. 

i The TV manipulation unit 13202 accepts the channel 

manipulation of the user, and transmits the same to the TV 
control unit 13203. ! 

The TV control unit 13203 receives the manipulation 
signal from the TV manipulation unit 13203, and in response 
carries out the action as a TV receiver , while sending a 
direction to the GIS generation unit 106 to generate the 
graphical interactive screen. J 

The TV GIS structure data storage unit 13204 stores in 
advance the GIS structure data as the TV receiver 1 s functions . 
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The 5 GIS structure data are almost identical with the objects 
shown in FIG. 108 through 117 in the tenth embodiment, and the 
explanation thereof is omitted. 

The GIS generation unit 10615 receives the generation 
direction from the TV control unit 13203, and in response 
retrieves the data from the TV GIS structure data storage unit 
13204, and generates the interactive screen by synthesizing the 
graphical display elements stored in the graphical display 
elements storage* unit 10613 as per the retrieved data. 
Accordingly the graphical display unit 10616 displays the 
graphical interactive screen thus generated. 

When the GIS generation unit 10615 receives the 
manipulation direction from the manipulation direction data 
receipt unit 10614, it generates the graphical interactive 
screen by synthesizing the graphical display elements in the 
graphical display element storage unit 10613 as per the GIS 
structure data in the display's GIS structure, data storage unit 
10612. Accordingly, the graphical display unit 10616 displays 
the graphical interactive screen thus generated. 

In this embodiment, the graphical interactive screen 
display function is explained as one of j the TV receiver 
» functions, and since the operation related to this embodiment 
is almost identical with that of the tenth embodiment, the 
further explanation is omitted. 
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THIRTEENTH EMBODIMENT 
FIG. 133 is a view depicting the structure of a 
graphical interactive screen (GIS) display direction unit in 
accordance with the thirteenth embodiment of the present 

i 

invention. The GIS display direction unit comprises a RM unit 
13301 and a GIS display unit 13201 • The GIS display unit 13201 
is identical with the GIS display unit 13201 of the twelfth 
embodiment, and thus the explanation is omitted. The RM unit 
13301 corresponds to the appliance to direct the display of the 
.graphical interactive screen in the tenth embodiment. The RM 
"unit 13301 comprises a user's GIS structure data storage unit 
13302, a GIS structure data transmission unit 13303, a 
. Manipulation unit 13304, a control unit 13305, and a 
manipulation direction data transmission unit (RM transmission) 
unit 13306. 

The user's GIS structure data storage unit 13302 stores 
in advance the GIS structure data for specifying the type and 
kind of the VTR to be controlled. The GIS structure data are 
identical with the data shown in FIGS. 108 through 117 except 
that the candidate values of the display candidate attribute, 
and the explanation thereof is omitted. 

The IS structure data transmission unit 13303 receives 
the transmission direction from the control unit 13305, and in 
response retrieves the GIS structure data from the user's GIS 

t 

structure data storage unit 281 to transmit the same to the GIS 

i 

structure data receipt unit 10611 via the transmission path 
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10623. 

The manipulation unit 13304 includes the control buttons 
fdr the VTR besides the control buttons for TV receiver. When 

i 

'"the user pushes one of the buttons, a signal assigned to that 

* .j 
\ i 

tmttbn is transmitted to the control unit 13305. 

The control unit 13305 initiates the RM set control 
action for the VTR when one of the VTR control buttons is 
pushed, while sending a transmission direction to the GIS 
structure data transmission unit 13303, generating the event 
signal according to the pushing of the button. The event 
signal thus generated is transmitted to the RM signal 
transmission unit 13306 for further transmission. 

; The RM transmission unit 13306 transmits the event 
signal generated by the control unit 13305 to the manipulation 
direction data receipt unit (RM receipt unit) 10614 via the 
transmission path 10628* 

Since the operation related to this embodiment is almost 
identical with that of the tenth embodiment, and a brief 
explanation will be given. When the user pushes the hutton 
of the manipulation unit 13304 on the RM unit 13301, then the 
RM unit 13301 originally furnished for controlling the TV 
receiver can also control the VTR. FIG. 134 is an example of 
the graphical interactive screen shown in the graphical display 
unit 10616 of the GIS display unit 13201, which shows the VTR 
is made by the Matsushita Electric Industrial Company in 1989. 
Thus, the user pushes the down button of the, manipulation unit 
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13304 twice, "then the "end button" exhibits 

Further, the user pushes the "determination button", then, 

"Matsushita" and "1989" are saved, enabling the RM unit 13301 

to* control this VTR. 
■/ 

>*/ In this embodiment, ' The user's GIS structure data 

storage unit 13302 of the RM unit 13301 stores the GIS 
structure data for controlling the VTR. However, since it is 
always used with the TV receiver (GIS display unit 13201) as 
shown in FIG. 135, the GIS structure data storage unit 13501 
may be additionally furnished in the graphical screen display 
unit 13201 to store the GIS structure data for controlling the 
VTR, and the user's GIS structure data storage unit 13302, GIS 
structure data transmission unit 13303, GIS structure data 
receipt unit 10611, and transmission path 10623 may be deleted. 
In this case, the GIS generation unit 10615 retrieves the GIS 
structure data for the VTR control from the GIS structure data 
storage unit 13501 upon receipt of the event signal by the 
manipulation direction data receipt unit 10614. 

FOURTEENTH EMBODIMENT 

FIG. 136 is a view depicting a GIS display direction 
unit in accordance with the fourteenth embodiment of the 
present invention. 

The GIS display direction unit comprises a GIS display 
unit 10610, a VTR 13601, and a RM unit 13602 for the VTR 13601. 

The GIS display unit 10610 is identical with the one in 
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the tenth embodiment, and the explanation is omitted. 

The VTR 13601 includes a RM receipt unit 13603, and a 
user's GIS structure data receipt unit 13604 in addition to the 
structure of the VTR 10620 of the tenth embodiment. 

The RM unit 13602 is .a remote controller of the VTR 
f 13061, and comprises a RM's- GIS structure data storage unit 
13605, a RM's GIS structure data transmission unit 13606, a 
manipulation unit 13617, a' control unit 13608, and a RM 
transmission unit 13609. 

The RM's GIS structure data transmission unit 13616 and 
the user's GIS structure data receipt unit 13604 are connected 
by a transmission path 13610, while the RM transmission unit 
13619 and RM receipt unit 13603 are connected to a transmission 
path 13611. 

The RM unit 13602 is almost identical with its 
counterpart in the thirteenth embodiment, except that it does 
not include the TV receiver's function control. Since the RM 
unit 13602 is used exclusively for the VTR 13601, it can not 
transmit any signal to the GIS display unit 10610. 

The control unit 13608 receives the signal by pushing 
the button of the manipulation unit 13607 by the user, and in 
response initiates the RM set control action, while sending a 
transmission direction to the RM's GIS structure data 
transmission unit 13606 to transmit the GIS structure data in 
the RM's GIS structure data storage unit 13605 to the user's 
GIS structure data receipt unit 13604 of the VTR 13601, 
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generating the event signal to transmit the same to the RM 
transmission unit 13609. 

The user's GIS structure data transmission unit 13604 
of the VTR 13601 receives the GIS structure data in the RM's 
GIS structure data storage unit 13605 of the RM unit 13602 from 
the RM f s GIS structure data transmission unit 13616, and in 
response, transmits the same to the GIS display unit 10610 by 
means of the GIS structure data transmission unit 10621. 

The RM receipt unit 13603 receives the event signal from 
the RM transmission unit 13609, and transmits the same to the 
GIS display unit 10610 by means of the user's direction data 
transmission unit 10624. 

The operation related to this embodiment is identical 
with that to the tenth embodiment except that the GIS structure 
data and manipulation signal from the RM unit 13602 are 
transmitted to the GIS display unit 10610 by means of the VTR 
13601, and the further explanation is omitted. 

In this embodiment, the RM's GIS structure data storage 
unit 13605 of the RM unit 13602 stores the GIS structure data 

i 

in advance. However, as shown in FIG. 137,, a GIS structure 
Sata storage unit 13701 storing the RM 1 s GIS structure data may 
be furnished in the VTR 13601, so that the control unit 13608 
of the RM unit 13602 sends a transmission direction of the RM's 
GIS structure data to the VTR 13601 instead by sending the 
transmission direction to the RM's GIS structure data 
transmission unit 13606. Upon receipt of the transmission 
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direction, the GIS generation unit 10615 retrieves the RM's GIS 
structure data from the RM's GIS structure data storage unit 
13701, and transmits the same to the GIS display unit 10610. 

I 

^According to this construction, the RM's GIS structure data 
/storage unit 13605, RM's GIS structure data transmission unit 
13666 in the RM unit 13602, the user's GIS structure data 
receipt unit 13604 in the VTR 13601, and the transmission path 
13610 can be omitted. 

In the tenth through fourteenth embodiments, the VTR was 
: used as an example of the appliance that directs the display 
of the graphical interactive screen. However, the appliance 
may be a movie player, air conditioner, audio apparatus, LD 
player, CD player, karaoke player, microwave oven, washing 
machine, facsimile, etc. 

FIFTEENTH EMBODIMENT 

FIGS. 138 and 139 are views depicting the structure of 
a menu information synthesis apparatus in accordance with the 
fifteenth embodiment of the present invention. The menu 
information synthesis apparatus comprises a remote controller 
13801, a TV 13802 and a VTR 13803 which are i controlled by RM 
'/signals transmitted from the remote controller 13801. 

The TV 13802 includes a TV menu information storage unit 
13821, a TV RM-signal receipt unit 13822, a TV control unit 
13823, and a TV menu information transmission unit 13824. 

The TV menu information storage unit 13821 stores TV 
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menu information and is composed of a TV menu structure data 
storage element 13911, a TV display element data storage 
element 13912, and a TV synthesis direction data storage 
element 13913. The menu information referred herein includes 
menu structure data, display element data and synthesis 
direction data. 

The TV menu structure data storage element 13912 stores 
the menu structure data as shown in FIGS. 140 and 141; the menu 
structure data are the data specifying the structure of a menu 
screen. 

The TV display element data storage element 13912 stores 
the display element data. The display element data are shape 
data for objects such as buttons and panels that form the menu 
screen (hereinafter referred to as the menu objects). As shown 
in FIG. 145, the display element data include various shape 
data depending on the selection of the menu objects. For 
example, a button shape A14501 shows that a button is selected, 
and a button shape B14502 shows that a button is not selected. 
The shape data include bit map data and/or graph data, and the 
TV display element data storage element 13912 stores the image 
data in relation with their respective identifiers. 

The TV synthesis direction data storage element 13913 

i 

stores synthesis direction data. The synthesis direction data 
are rules that regulate the synthesis of the menu structure 
data and display element data by combining a plurality pieces 
of menu structure data and display element data. As shown in 
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FIG. 146, the synthesis direction data include a rule "TOP 
synthesis" for synthesizing the menu structure data, and a rule 
"PRIORITY" for synthesizing the display element data. 

Note that the menu structure data and synthesis 
5 direction data are written using the grammar of SGML arranged 
by the object oriented technology. Each menu object is 
described as an object and belongs to a class. 

The class consists of an entity class, a panel class, 
a button class, and a page button class, and these classes have 

10 hierarchical correlation with each other as shown in FIG. 147. 

Next, the menu structure data in the TV menu structure 
data storage element 13912 will be explained. 

FIG. 140 provides a class definition. For example, the 
button class has shape, action, parent class attributes, and 

15 the default values of the shape and action are a button shape 
and a button action respectively and the parent class of the 
button class is the entity class. The class attribute values 
for the shape and action are inherited to the object by the 
"class inheritance" based on the object oriented technology. 

20 That is to say, if the attribute value of an object is defined 
by the object, the defined attribute value is valid, and if it 
is not defined by the object, the attribute value of a class 
to which the object belongs becomes valid. If the attribute 
value is not defined in the object's class either, the 

25 attribute value of another class higher than the object's class 
becomes valid. If the attribute value is not defined in the 
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higher class, then the attribute value of another class higher 
than the last mentioned class becomes valid. In this way, the 
attribute value is checked in the classes in a lower- to-higher 
sequence, and a firstly-found attribute value becomes valid for 
the subject object. The button shape is defined as shown in 
FIGl 143. When the selection state is ON, the button shape is 
the shape data named as a button shape A, while it is the shape 
data named as a button B when the selection state is OFF. The 
shape data of the "button shape A14501 and button shape B14502 
are stored in the TV display element data storage element 13912 
as shown in FIG. 145. Note that "the selection state is ON" 
means that the button is pushed (selected), and "the selection 
state is OFF" means that the button is not pushed. Thus, a 
button appears as the button shape B14502 in general, and it 
appears as the button shape A14501 only when it is pushed. 

The definition of the panel class is given in FIG. 140, 
and its default shape is the panel shape, and the default 
character string is a menu screen. The attribute value of the 
character string attribute is an index of a panel placed at 
the highest position. 

FIG. 141 defines the panel object.; The information 
sandwiched by <PANEL> and </PANEL> is the definition of the 
panel. For example, the set panel includes an image adjustment 
button, a speech adjustment button, a BS input level set 
button, a receiver channel set button, a previous page button, 
and a next page button. Also, <v>, </v>, <h>, and </h> are the 
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information related to the position of the objects- if the 
objects are sandwiched by <v> and </v>, they are placed 
vertically, and if the objects are sandwiched by , <h> and </h>, 
they are placed horizontally. Thus, on the set panel, the 
previous page button and next page button are placed 
horizontally, and the image adjustment button, speech 
adjustment button, BS input level set button, receiver channel 
set button, and the horizontally placed previous page button 
and next page button are placed vertically. 

Next, the definition of the action will be explained 
while referring to FIG. 142. 

For example, there is "ACTION = SET_BUTTON_ACT I ON" in 
the set button definition, and the set button action is defined 
in FIG. 143. The action definition controls the action 
(transmission message and the destination of the transmission) 
executed when a corresponding button is pushed. Thus, the set 
button action defines the action when the set button is pushed: 
a^ message "open panel" is transmitted to the "set panel". 
Thus, the "set panel" appears on the screen when the set button 
action is executed. 

The TV synthesis direction data shown in FIG. 146, 
" SYNTHES I S_D I RECT I ON COMPUTATION = TOPJSYNTHESIS COMBINE^ 
METHOD = UNIQ__COMBINE> will be explained in the following. 

The computation "COMPUTATION - TOP_SYNTHESIS" means to 
synthesize the menu structure data in the panels identified by 
an identifier "top panel" in the definition of the panel object 
j 
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shown in FIG. 141 • The top panel is the panel placed at the 
highest position in the hierarchical structure, in other words, 
the panel appears first when the menu is displayed. 

" COMB I ne__method * uniq_combine" means to make the same 
objects such as buttons in the menu structure data of the top 
panels into one object when synthesizing the menu structure 
data. 

"</SYNTHESIS_DIRECTION> represents the end of one 
synthesis direction. 

As has been stated, a menu screen of one hierarchical 
structure can be made when synthesizing a plurality of 
hierarchical menu screens by the TOP synthesis. 

Next, "<SYNTHESIS_DIRECTION COMPUTATION - PANEL_ 
SYNTHESIS COMBINE J^IETHOD - UNIQ_COMBINE> iwill be explained. 

i 

"COMPUTATION = PANELJ3YNTHESIS" means to synthesize a 
plurality of panels having the same identifier. 

"C0MBINE_METH0D = UNIQ_COMBINE " defines a method to 
combine a plurality of panels. "UNIQ_COMBINE" means to allow 
only one object (button or the like) for one identifier within 
5a plurality of panels to be synthesized* This eliminates 
overlapping objects which are assumed to have the same 
function. i 

Next, the synthesis data " <SYNTHESIS_DIRECTION 
COMPUTATION = PRIORITY> will be explained. 

"COMPUTATION = PRIORITY" means to give a priority to the 
menu structure data of TTT identified by "APPLIANCE = TTT" 

158 



Sandwiched by <SYNTHESIS_DIRECTION> and </SYNTHESIS__DIRECTION> 
over the menu structure data transmitted from the other 
appliances. In case of FIG. 146, the shape that the menu 
structure data for the TV 13802 includes is given priority to 
be used for the button shape and page button shape, making a 
straightforward synthesized menu screen. 

The TV RM-signal receipt unit 13822 receives a 
transmission signal from the remote controller 13801, and 
transmits the same to the TV control unit 13823. 

The TV control unit 13823 receives the signal from the 
TV RM signal receipt unit 13822, and checks whether the 
received signal is a menu-information transmission request 
signal or action control signal. In case of the former, the 
TV control unit 13822 gives a direction! to the TV menu 
information transmission unit 13824 to transmit the menu 
information, and in case of the latter, it controls the action 
of * the TV 13802 as per received signal. ; 

f The TV menu transmission unit 13824 receives the menu- 

information transmission direction from the TV control unit 

13823, and in response retrieves the TV menu information from 

i 

the TV menu information storage unit 13821 to send the same to 

j 

the remote controller 13801. ] 

i 

The VTR 13803 includes a VTR menu information storage 
unit 13831, a VTR RM-signal receipt unit 18832, a VTR control 
unit 13833, and a VTR menu information transmission unit 13834. 

The VTR menu information storage unit 13831 stores the 

159 



menu information related to the VTR 13803 and it includes a VTR 
menu structure data storage element 13921 , a VTR display 
element data storage element 13922, and a VTR synthesis 
direction data storage element 13923. 

The VTR 13803 and TV 13802 are of the same structure 
except that the VTR menu information storage unit 13831 stores 
the menu information related to the VTR 13803 , and only the 
menu information related to the VTR 13803 will be explained. 

The VTR menu structure data storage element 13921 stores 
the menu structure data related to the VTR , 13803 as shown in 
FIGS. 140, 148 through 151. 

The VTR display element data storage element 13922 

I 

stores the display element data related to the VTR 13803 as 
shown in FIG. 152. 

I 

The VTR synthesis direction data storage element 13923 

! 

stores the synthesis direction data related to the VTR 13803 
as shown in FIG. 153. ! 

The synthesis direction data "COMPUTATION = PRIORITY" 

i 

means to give a priority to the menu structure data sandwiched 
by <SYNTHESIS__DIRECTION> and </SYNTHESIS_DIRECTI0N> as 
previously explained: the shape in the menu structure data 
related to the VTR 13803 is given priority for the panel shape. 

"COMPUTATION = ADD" means to add additional information 
* when synthesizing the menu structure data, and "MESSAGE^ 
TRANSMISSION", or the value represented by "<SUBJECT>" is a 

unit of the additional information. The " MESSAGE__TRANSMISS ION" 

./ 
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information is composed of a transmission message and 
information as to the correspondence of the message. 

"CORRESPONDENCE = VTR_CONTROI»" specifies a subject to 
be added, or namely the " MES S AGE_TRANSMI SS I ON " information. 
In other words, when there exists " MESS AGEJTRANSMISS ION " 
information that matches ■ with the " CORRESPONDENCE <VTR__ 
CONTROL>", the specified information is added at a specified 
position. " <ADD_DESTINATION> BEFORE" specifies the position of 
the additional* information. In this embodiment, 

"CORRESPONDENCE . = VTR__CONTROL " means to add the "additional 
information" as a pre-process for the "MESSAGEJTRANSMISSION" 

information that matches with the " CORRESPONDENCE <VTR_ 

S 

CONTROL>". Note that there is " <ADDJDESTlkATION> AFTER", as a 
definition of a post-process besides " <ADD_DESTINATION> BEFORE" . 

" <ADD_INFORMATION MESSAGE » VTR_SWITCH 
CORRESPONDENCE = TVJ30NTR0L> MESSAGE JTRANS^ISS ION" specifies 
the unit of the additional information and the content thereof. 
In this embodiment, it means to add the additional information 
related to "MESSAGE_TRANSMISSION" , and : the content is 
"<fMESSAGE>VTR_SWITCH" and " < CORRESPONDENCE >TV_CONTROL" . Thus, 
"<SYNTHESIS_DIRECTION COMPUTATION = ADD> and the following mean 
to add the information of the message transmission of 
"<MESSAGE>VTRJ3WITCH" and " <C0RRESP0NDENCE>TVJ20NTR0L" as the 
pre-process information when there exists the "MESSAGE_ 
TRANSMISSION" information that matches with " <CORRESPONDENCE> 
VTR CONTROL" . 



The remote controller 13801 includes an input acceptance 
unit 13811, a RM signal transmission unit 13812, a menu 
information receipt unit 13813, a menu information storage 
unit 13814, a display element data storage unit 13815, a menu 
information synthesis unit 13816, a menu information display 
Control unit 13817, and a menu display unit 13818. 

The input acceptance unit 13811 comprises a transparent 
or semi -transparent touch panel placed on the menu display unit 
13818, and an input button furnished fixedly, and accepts a 
user's input manipulation. By the push of the input button, 
the input acceptance unit 13811 accepts 1 the transmission 
request of the menu information related to the TV 13802 or VTR 
13803, and notifies the same to the RM signal transmission unit 

i 

13812. Similarly, the input acceptance unit 13811 accepts the 
synthesis direction of the menu information; by the push of the 
input button, and notifies the same to the menu information 
'synthesis unit 13816. Also, the input acceptance unit 13811 
accepts the manipulation to the touch panel (the push on the 
displayed menu objects on the menu display unit 13818); in 
case of a menu display change, it sends a change direction for 
the menu objects to the menu information display control unit 
13817, and in case of an action direction to the TV 13802 or 
VTR 13803, it transmits a control signal assigned to the 
manipulated menu object to the RM signal transmission unit 
13812. 

The RM signal transmission unit 13812 receives the menu- 
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information transmission request from the input acceptance unit 
13811, and in response transmits a RM signal requesting the 
/menu-information transmission to the TV 13802 or VTR 13803. 
Also, when the RM signal transmission unit 13812 receives the 
5 t control signal from the input acceptance unit 13811, it 
\/ converts the control signal into the RM signal to transmit the 
same to the TV 13802 or VTR 13803. 

The menu information receipt unit 13813 receives the 
menu information related to the TV 13802 and VTR 13803 from the 
f 10 TV menu information transmission unit 13824 and VTR menu 
information transmission unit 13834 respectively, and in 
response stores the same separately into the menu information 
storage unit 13814. 

The menu information storage unit 13814 stores the menu 
15 information received by the menu information receipt unit 
13813. 

The display element data storage unit 13815 stores in 
advance the display element data that the remote controller 
13801 has. The display element data referred herein are the 

20 default data used when the received menu information does not 
include the display element data to synthesize a menu screen 
upon receipt of the menu information. 

The menu information synthesis unit 13816 includes a 
retrieval check unit, a synthesis direction count unit, a TOP 

25 synthesis unit, a panel synthesis unit, a PRIORITY unit, an ADD 
unit, and a temporarily storage unit. 
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The retrieval • check unit receives the synthesis 
direction from the input acceptance unit 13811, and in response 
retrieves the menu information from the menu information 
storage unit 13814 to store the same in the temporarily storage 

> ? 5 • unit, checking whether there exists the menu information for 

/■ 

a plurality of appliances. If so, the retrieval • check unit 
gives a direction to set an initial value to the synthesis 
direction count unit; otherwise, it sends a notice of 
completion of synthesis to the menu information display control 

i 

10 * unit 13817* Further, the retrieval * check unit checks whether 
the menu information includes the synthesis direction data or 
not* If the menu information includes the synthesis direction 
data, the retrieval • check unit retrieves the N f th synthesis 
direction data specified by a counter , in the synthesis 

15 direction count unit to check whether "XXX" in "COMPUTATION « 
XXX" in the retrieved synthesis direction data is the "TOP 
synthesis" computation, "panel synthesis" computation, 
"PRIORITY" computation, or "ADD" computation, or none of the 
aforementioned* In case of the "TOP synthesis" computation, 

20 the retrieval *check unit activates the TOP synthesis unit. 
Similarly, in case of the "panel synthesis" computation, 
"PRIORITY" computation, and "ADD" computation", it activates 
the panel synthesis unit, PRIORITY unit, and ADD unit, 
respectively* If "XXX" is not any of the aforementioned, it 

25 gives a direction to the counter of the synthesis direction 
count unit to increment by one. 
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The synthesis direction count unit counts the number of 
the synthesis direction data. When it receives the direction 
to set the initial value from the retrieval • check unit, it sets 

4 

9. 

N=l to the counter, and increments the counter by one each time 
it: receives an increment: direction from the retrieval • check 
unit, TOP synthesis unit, panel synthesis unit, PRIORITY 
synthesis unit or ADD unit. 

The TOP synthesis unit is activated by the 
retrieval •check 'unit, and checks whether "YYY" in "COMBINE^ 
METHOD=YYY " is either "UNIQ_COMBINE n or "SIMPLE_COMBINE" , or 
neither. In casfe of either " UN I Q_COMB I NE" or "SIMPLEJ30MBINE" , 
it sets a value "1" to an appliance counter M installed therein 
(M«l), and successively retrieves the menu structure data from 
the temporarily storage unit, while incrementing the appliance 
counter by one for each retrieval. The TOP synthesis, unit 
retrieves the definition data of the panel 1 identified by the 
"top panel" in the M f th appliance's menu structure data, and 
adds the attribute definition in the definition data of the 
retrieved top panel to the menu structure data in the 
temporarily storage unit as a definition item of the attribute 

' of a synthesized panel. The attribute definition referred 
herein is "aaa=(3pp • ♦ ♦ " in " < PANEL aaa=pp(3 ♦ • •TOP_PANEL" . It 

'adds the objects sandwiched by " <PANEL> " and " </PANEL>" in the 
definition data of the panel identified by the "top panel" in 
the menu structure data of the M'th appliance as the objects 
sandwiched by 11 <PANEL> " and " </PANEL> " of the synthesized 
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panel . If "YYY" is neither "UNIQ_COMBINE" nor "SIMPLE^ 
COMBINE" , it gives a increment direction to the counter of the 
synthesis direction count unit. 

For example, the menu information storage unit 13814 
V 5 -stores the menu structure data of the TV 13802 as shown in 
FIG- 141, and the menu structure data of the VTR 13803 as shown 
in FIG. 148, then they are synthesized into the menu 
information (top panel) as shown in FIG* 154: the attribute 
definition of the VTR 13803 is simply added to the attribute 
10 definition of the TV 13802, and all the objects for the VTR 
13803 are placed following to the lowest-position objects of 
the TV 13802, 

When there are overlapping attribute definition items 
in the synthesized panel, the TOP synthesis unit gives a 

15 priority to the first-written definition item, and deletes the 
rest from the temporarily storage unit, completing "SIMPLE 
COMBINE" . ; 

Further, the TOP synthesis unit checks whether "YYY" is 
"UNIQ_COMBINE" or not. In case of "UNIQ_C0MBINE" , when there 

20 are overlapping objects between " <PANEL> " and "</PANEL> in the 
synthesized panel, the TOP synthesis unit deletes the 
overlapping objects from the temporarily storage unit, and also 
arranges the position information "<v>" and M </v"> in 
accordance with its nature. If "YYY" is not "UNIQ_COMBINE" , 

25 the TOP synthesis unit gives an increment direction to the 

./ counter of the synthesis count unit. 

.< 
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For example, the simply synthesized top panel as shown 
in FIG- 154, is changed to the one as shown in FIG. 155 by 
deleting the overlapping attribute definition "CHARACTER_ 
STRING=VTR_MANIPULATION_MENU" . Further, in FIG. 155, the 
*5 overlapping objects " <BUTTON>SET_BUTTON" are deleted as shown 
in FIG. 156, and the position information, " <v> and </v>, is 
■ arranged as shown in FIG. 157. In this way/ the top panels are 
4 synthesized into one panel. 

The panel synthesis unit is activated by the 
10 retrieval -check unit, and checks whether " YYY " in "COMBINE^ 
METH0D=YYY " is either " UN I Q_C0MB I NE 11 or " SIMPLE_COMBINE" , or 
neither. In case of either "UNIQ_COMBINE" or "SIMPLE_C0MBINE\ 
the panel synthesis unit retrieves the menu structure data from 
the temporarily unit to compute the number of sets of the 
15 panels, each set having the same identifier in the menu 
structure data. In other words, there are three sets when the 
menu information related to the TV 13802 includes the panels 
identified by the identifiers "panel 11"', "panel 12", and 
"panel 13", and the menu information related to the VTR 13802 
20 includes the same, and there are no other overlapping panels 
identified by the same identifier. If "YYY" is neither "UNIQ_ 
COMBINE" nor "SIMPLE_C0MBINE" , the panel synthesis unit gives 
an increment direction to the counter of the synthesis 
direction count unit. Assume that K represents the number of 
25 the sets, and a set-number counter K 1 is furnished in the panel 
synthesis unit, then the panel synthesis unit checks whether 
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•the value in the set-number counter K 1 is greater than zero, 
(K ! > 0, where initial K 1 = K ) . If so, it retrieves all the 
menu structure data related to a plurality of panels including 
the overlapping panel identifiers in the K f th set to compute 
the number (L) of the panels within the set, setting the* same 
in a panel number counter. Otherwise, it gives an increment 
direction to the counter of the synthesis direction count unit. 
Further, the panel synthesis unit checks whether L > 0. If so, 
it adds all the attribute definitions of the L'th panel 
information to the temporarily storage unit as the attribution 
definition items of the synthesized panel. The panel synthesis 
unit adds the objects defined between "<PANEL>" and " < /PANEL> 
in the L'th panel information to the temporarily storage unit 
as the objects between "<PANEL>" and "</PANEL> in the 
synthesized panel, decrementing the panel number counter by 
one. Otherwise, when there are overlapping attribute 
definition items in the synthesized panel, the panel synthesis 
unit gives a priority to the first-written attribute definition 
and deletes the rest from the temporarily storage unit. 
Further, the panel synthesis unit checks whether "YYY" is 
"UNIQ__COMBINE n or not. In case of "UNIQ__COMBINE M , the panel 
synthesis unit deletes the overlapping objects between the 
" <PANEL> " and 11 < /PANEL> " in the synthesized panel and 
decrements K in the set number counter. For example, if the 
menu information storage unit 13814 stores the menu structure 
data (set panel) for the TV 13802 as shown in FIG. 141, and the 
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menu structure data (set panel) for the VTR 13803 as shown in 

FIG. 148, they are simply synthesized as shown in FIG. 158. 

The objects defined in the same lower position are deleted as 

shown in FIG. 159, and further the position information, "<v>", 

"</v>", "<h>", and "</h>", is' arranged as shown in FIG. 150. 

The PRIORITY unit is activated by the retrieval • check 

upit, and retrieves an appliance attribute value "appliance = 
« 

MMM" from the synthesis direction data. The PRIORITY unit 
retrieves the shape information sandwiched by " <SYNTHESIS_ 
DIRECTION> " and " </SYNTHESIS JDIRECTION> " , , and updates the 
content of the temporarily storage unit by rewriting the shape 
attribute of the appliances other than the appliance " MMM 11 to 
the shape attribute of the appliance "MMM" . 

For example, in "PRIORITY" computation shown in FIG. 
146, the attribute value defined for the TV 13802 is given 
1 priority to the button shape and page button shape over the 
other menu structure data (menu structure data of the VTR 
13803). Similarly, in "PRIORITY" computatibn in FIG. 153, the 
attribute value defined for the VTR 13803 is given priority 
and used for the panel shape. In this way, as shown in FIG. 
161, the post-synthesis shape data uses the button shape and 
page button of the menu structure data of the TV 13802 and the 
panel shape of the menu structure data of the VTR 13803. 

The ADD unit is activated by the retrieval • check unit, 
and retrieves "SUBJECT BBB=CCC • • • > DDD " from the synthesis 

direction data in the temporarily storage unit, and searches 
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f6r the information identified by "BBB=CCC • • • M in "DDD 11 • If 
"BBB=CCC • • • " contains a plurality of conditions, it searches 
for the information that satisfies all the conditions and 
.numbers them sequentially if there are plurality of them* The 
AQD unit sets a value "1" to an add number counter p (p=l). 
Also, the ADD unit checks whether there are the p'th retrieved 
data, and if so, it adds "FFF=GGG* • • " in " <ADDITIONAL 
INFORMATION FFF = GGG • • ♦ >HHH" at a position identified by 
"EEE" in "<ADD DESTINATION>EEE" in the synthesis direction data 
to the p'th retrieval data as the menu structure data. The ADD 
unit increments the add number counter p by one. If there are 
, no p'th retrieval data, the ADD unit gives an increment 
direction to the counter of the synthesis direction count unit. 

For example, in "ADD" synthesis shown in FIG, 153, the 
action to switch the control to the VTR 13803 as a pre-process 
for the control action over the VTR 13803 is added. That is 
to say, the control is switched to the VTR 13803 first, and 
then the action of the VTR 13803 is controlled. 

The temporarily storage unit stores the menu information 
under the control of the retrieval control unit, and adds, 
deletes and amends the content thereof under the control of the 
TOP synthesis unit, panel synthesis unit, PRIORITY unit and ADD 
unit. 

The menu information display control unit 13817 controls 
the menu display unit 13818 to display a menu screen. The menu 
information display control unit 13817 receives a notice of 
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completion of synthesis from the menu information synthesis 
unit 13816, and in response displays the synthesized top panel 
in the temporarily storage unit by means of the menu display 
unit 13818. The menu information display control unit 13817 
receives a direction to change the display screen from the 
input acceptance unit 13811, and in response displays the 
synthesized menu information in the temporarily storage unit 
in the menu information synthesis storage unit 13816 as per 
change direction 

Assume that the menu information storage unit 13814 
stores only the menu information for the TV 13802 as shown in 
FIGS. 140 through 145, then the menu information display 
control unit 13817 controls the menu display unit 13818 to 
display the menu screen, or a TV manipulation menu 16201 as 
shown in FIG. 162. When the set button 16202 is pushed as the 
menu object in the input acceptance unit 13811, it appears as 
the shadowed button shape A. The menu information display 
control unit 13817 receives the change direction from the 'input 
acceptance unit 13811, and in response retrieves the menu 
structure data from the menu information synthesis unit, 
controlling the menu display unit 13818 to display a set panel 
16203. The menu information display control unit 13817 adjusts 
the set panel 16203 into an adequate size to display the menu 
objects such as a button 16204 and a page button 16205. 

When the menu information storage unit 13814 stores only 
the menu information for the VTR 13803 as shown in FIGS. 140, 
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148 -through 151 , then the menu information display control unit 
13817 controls the menu display unit 13818 to display a VTR 
manipulation menu 16301 as shown in FIG- 163. Under these 

circumstances, the set panel 16303 is displayed when the set 

* .* 

•< 

button 16302 on the VTR manipulation menu 16301 is pushed. 

The menu display unit 13818 is composed of an LC display 
or the like, and displays the menu screen under the control of 
the menu information display control unit 13817. 

The menu display 13818 displays the menu structure data 
(top panel) shown in FIG. 157 as a menu screen shown in FIG. 
164 when the menu information for the TV 13801 and VTR 13802 
is synthesized by the menu information synthesis unit 13816. 
Also, it displays the menu structure data (set panel) shown in 
FIG. 160 as the menu screen shown in FIG. 165. 

Next, the operation related to this embodiment will be 
explained while referring to the flowcharts in FIGS. 166 
through 168. 

The remote controller 13801 transmits a signal to the 
TV 13802 (S16602), and the TV RM-signal receipt unit 13822 
receives the signal from the remote controller 13801 (S16604), 
and transmits the same to the TV control unit 13823 (S16606). 
The TV control unit 13823 checks whether the signal from the 
RM signal receipt unit 13822 is a signal requesting the menu- 
information transmission or not (S16608). In case of the menu- 
information transmission request signal, the TV control unit 
13823 gives a direction to transmit the TV menu information in 
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•the* TV menu information storage unit 13821 to the TV menu 
transmission unit 13824 (S166610). Otherwise, it proceeds to 
SJL6618. 

» 

The TV menu information transmission unit 13824 
retrieves the menu information to transmit the same to the 
remote controller 13801 (S16612), and the menu information 
receipt unit 13813 in the remote controller 13801 receives the 
TV menu information (S16614). The menu information receipt 
unit 13813 has the menu information storage unit 13814 store 
the TV menu information (S16616), and the TV control unit 13823 
controls the action of the TV 13802 as per signal from the TV 
control unit 13823. 

The remote controller 13801 and VTR 13803 operate in the 
same manner as above, and the explanation is omitted. 

The retrieval* check unit in the menu synthesis* unit 
1,3816 checks whether the menu information storage unit 13814 
stores the menu information for a plurality of appliances 
(S16702). If the menu information storage unit 13814 stores 
the menu information for a plurality of appliances, the 
retrieval • check unit gives a direction to set an initial value 
to the synthesis direction count unit, which accordingly sets 
N«l (S16704). Otherwise, it ends the synthesis process. 

The retrieval • check unit further checks whether there 
exist the N'th synthesis direction data in the synthesis 
direction data in the menu information storage unit 13814, and 
if there are the N'th synthesis direction data, it retrieves 
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the same; otherwise it ends the synthesis process (S16706). 

The retrieval • check unit interprets "XXX" in 
"COMPUTATION=XXX" in the retrieved synthesis direction data 
tQ check whether "XXX" is the "TOP synthesis" computation or 
not (S16708). In case of the "TOP synthesis" computation, the 
retrieval * check unit activates the TOP synthesis unit, which 
in turn checks whether "YYY" in "COMBINE METHOD=YYY" is either 
."UNIQ COMBINE" or "SIMPLEJ30MBINE" or neither (S16710). ' When 
"XXX" is not * the "TOP synthesis" computation, the 
retrieval • check unit proceeds to S16728. If "YYY" is either 
M UNIQ_COMBINE" or " SIMPLE_COMBINE " , the retrieval • check unit 
sets a value "1" to the appliance number counter (M=l). If 
"YYY" is neither "UNIQ_COMBINE" nor "SIMPLE_COMBINE" , the 
retrieval • check unit proceeds to S16762. Subsequently, the 
retrieval • check unit checks whether there is the M'th 
appliance \s menu (S16714), and retrieves the definition data 
of the panel identified by the "top panel" in the menu 
structure data of the M'th appliance, adding the attribute 
definition in the definition data of the retrieved top .panel 
as the definition item of the attribute of the synthesized 
panel (S16716). When the menu information of the M'th 
appliance does not exist, the retrieval •check unit proceeds to 
S16722. 

The retrieval • check unit adds the objects defined 
between " <PANEL> " and "</PANEL>" in the definition data of the 
panel identified by the "top panel" in the menu structure data 
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of the M'th appliance as the objects sandwiched by " < PANEL > ,! 
and 11 < /PANEL> " in the synthesized panel (S16718). Then, the 
retrieval • check unit increments the appliance number counter 
and" returns to S16714 (S16720). If there are any overlapping 
a/ttribute definition items in the synthesized panel, the 
retrieval -check unit gives a priority to the first-appearing 
definition item and deletes the rest (S16722). 

The retrieval • check unit checks whether " YYY " is "UNIQ_ 
COMBINE" or not (S16724), and in case of "UNIQ__COMBINE" , it 
deletes the overlapping objects between the " <PANEL> " and 
" </PANEL>" in the synthesized panel. In other words, it 
interprets the objects as one object if they have the- same 
identifier (S16726), and proceeds to S16762; otherwise it 
proceeds to S16762. 

The retrieval 'Check unit checks whether the retrieved 
synthesis direction data are the "panel synthesis" computation 
or not, and in case of the "panel synthesis" computation, it 
activates the panel synthesis unit, which checks "YYY" in 
"COMBINE_METHOD=YYY" is either "UNIQ__COMBINE" or "SIMPLE^ 
COMBINE" or neither (S16730). If "XXX" is not the "panel 
synthesis" computation, the retrieval • check unit proceeds to 
S16742. If "YYY" is either "UNIQ__COMBINE" or "SIMPLE_COMBINE" , 
the retrieval • check unit computes the number K of the sets of 
the panels having the same identifiers in the menu information 
retrieved in S16702 (S16732), and checks if K>0 (S16734). If 
K>0, the retrieval • check unit retrieves all the menu structure 
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data for a plurality of panels within the K'th panel having the 
same identifier (S16736) and synthesizes the same (S16736); 
otherwise, it proceeds to S16762. The menu-structure-data 
synthesis action will be explained below in S16802 and onwards. 
The retrieval -check unit decrements K (K = K -1) (SI 6740) and 
returns to,S16734. 

The retrieval •check unit checks whether the retrieved 
synthesis data are the " PRIORITY" computation or not (S16742), 
and in case of the "PRIORITY" computation, it activates the 
PRIORITY unit, which accordingly retrieves the appliance 
attribute value "appliance=MMM" from the synthesis direction 
data (S16744). The PRIORITY unit retrieves the shape 
information sandwiched by " <SYNTHESIS_DIRECTION> and 
"</SYNTHESISJDIRECTION>" (S16746), and rewrites the shape data 
of the appliances other than the appliance "MMM" into the shape 
data of the appliance "MMM" (S16748) and proceeds to S16762, 

The retrieval* check unit judges whether the retrieved 
synthesis direction data are the "ADD" computation or not 
(S16750), and in case of the "ADD" computation, it activates 
the ADD unit, which accordingly retrieves "SUBJECT BBB=CCC 
• • • >DDD" from the synthesis direction data to search for the 
information identified by "BBB=CCC • • . " in the menu structure 
data "DDD" . If "BBB=CCC • • • " includes a plurality of 
conditions, the information that satisfies all the condition 
is Retrieved (S16752). Subsequently, the retrieval • check unit 
'sets a value "1" to the additional number counter (p=l) 



(S16754). The retrieval • check unit checks whether there are 
the p'th retrieved data in the " DDD " data received in S16752 
(S16756). If there are the p'th retrieved data, the 

retrieval • check unit adds the data " FFF=GGG • • • " in " < ADD 
INFORMATION FFF=GGG • * • >HHH M at a position specified by "EEE" 
in 11 ADD__DESTINAT ION> EEE 11 (S16758) . Subsequently, the 

. retrieval • check unit increments the additional number counter 
p by one (S16760), and returns to S16756. If there are no p'th 
retrieved data, the retrieval • check unit gives an increment 
direction to the counter of the synthesis direction count unit 
(S16762), and returns to S16706. 

The panel synthesis unit computes the number L of the 
panel having the same identifier (S16802), and checks whether 
L > 0 (S16804). In case of L > 0, the retrieval • check unit 
adds the attribute definition of the L f th panel as the 
attribute definition item of the panel to be synthesized 
(S16806). The retrieval • check unit adds the objects sandwiched 
by "<PANEL>" and " </PANEL>" in the L ! th panel information as 
the objects sandwiched by " <PANEL> " and "</PANEL>" in the 
synthesized panel (S16808), and decrements L by one (L = L - 
1) (S16810), returning to S16804. If L is not greater than 
zero, and there are overlapping attribute definition items in 
the synthesized panel, the panel synthesis unit gives a 
priority to the first-appearing attribute definition item, and 
deletes the rest (S16812). The panel synthesis unit checks 
whether " YYY " is "UNIQ_COMBINE" or not (S16814), and in case 
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of "UNIQ COMBINE", it deletes the overlapping objects 
sandwiched between " <PANEL> and </PANEL> " if any (S16816), 
returning to S16740. Otherwise, it returns to S16740. 

; Although the TOP synthesis unit writes the overlapping 

attribute definition from the second in the temporarily storage 
unit in S16722, the priority may be given to each appliance, 
so that the attribute definition of the appliance will be 
adopted in an order of priority. 

Also, the "top synthesis unit deletes the same object 
appearing at the second and subsequent S16726. However, the 
priority may be given to each appliance, so that the attribute 
definition of the appliance will be adopted in an order of 
priority* 

Further, in S16746, the PRIORITY unit retrieves the 
shape information to match the menu structure data with the 
shape data of a specific appliance. However, other attribute 
definitions such as character color, action, position, panel's 
timeout (the attribute value of the time from the start to end 
of the panel display) to match these attribute definitions. 

In this embodiment, the menu information for the TV 
13801 and VTR 13802 are synthesized. However, the appliances 
are not limited to these two appliances: it may include an air 
conditioner or audio equipment, etc. 

* Although the present invention has been fully described 
by way of example with reference to the accompanying drawings, 
it is to be noted that various changes and modification will 
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be apparent to those skilled in the art. Therefore, unless 
otherwise such changes and modifications depart from the scope 
of the present invention, they should be construed as being 
included therein. 
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